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ABSTRACT 


The  Operation  with  Economic  Efficiency  (OEE;  Program  in  the  Republic  of 
Korea  Air  Force  (ROKAF)  was  designed  to  reduce  the  cost  and  improve  the  efficiency 
of  all  ROKAF  operations.  The  ROKAF  is  now  faced  with  the  problem  of  developing 
a  more  efficient  pilot  candidate  selection  and  training  program  which  will  improve  pilot 
quality  and  reduce  attrition  rates  and  the  number  of  aircraft  accidents.  The  present 
research  provides  a  starting  point  for  the  development  of  a  more  efficient  pilot 
selection  and  training  program.  If  successful  this  program  should  result  in  the  selection 
of  pilot  candidates  exhibiting  a  significantly  higher  probability  of  completing  the  pilot 
training  program.  This  thesis  also  provides  a  summary  of  aviation  selection  test 
batteries  presently  in  use  by  the  L\S.  Air  Force  and  U.S.  Navy  and  applies  the 
principles  of  decision  theory  in  determining  whether  a  change  in  the  ROKAF's  present 
pilot  selection  system  is  feasible. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Throughout  the  history  of  aviation  there  has  been  constant  concern  with 
selecting  those  individuals  with  both  mental  and  physical  qualities  desirable  for 
optimum  performance  in  flight  environments.  The  intensity  and  quantity  of  research 
performed  over  the  years  in  the  particular  area  of  pilot  selection  highlights  the  fact  that 
the  key  to  combat  air  superiority  is  the  selection  of  high  quality  pilot  candidates. 
During  World  War  II  and  the  Korean  conflict  it  became  further  apparent  that  the 
ability  to  gain  total  tactical  and  strategic  advantage  over  the  enemy  depended  on 
whether  or  not  air  superiority  was  achieved.  Fortunately  the  L'.S.  was  able  to  achieve 
this  superiority  in  the  air  during  the  last  three  major  conflicts.  Unfortunately  there  is  a 
growing  consensus  among  military  analysts  that  the  L'.S.  may  be  lulling  itself  into  a 
sense  of  complacency.  The  following  quotation  from  a  publication  by  B.  K.  Holloway 
underscores  the  basis  for  this  growing  complacency:  [Ref.  1] 

•  "Twenty-six  million  living  Americans  are  veterans  of  military  service,  and  most 
of  them  have  served  in  wartime.  How  many  of  these  26  million  ever  had  to  face 
an  enemy  who  held  air  superiority  ?  ...  In  all,  probably  no  more  than  one  out 
of  150,  for  after  February  1943  the  L'.S.  and  allies  had  undisputed  air 
superiority  in  the  Mediterranean  and  Western  Europe;  after  August  of  1943  we 
had  it  in  the  Pacific...  In  Korea  we  won  air  superiority  twice  -  from  the  North 
Korean  Air  Force  in  the  first  two  months  of  that  war,  and  again  from  the 
Chinese  Air  Force  after  November  1950.  In  South  Vietnam,  our  air  superiority 
has  come  by  default.  In  North  Vietnam  it  has  yet  to  be  seriously  challenged." 

Today  we  have  reached  an  age  of  strategic  parity  where  Eastern  block 
technological  developments  in  air  combat  systems  are  advancing  at  a  rate  comparable 
with  those  as  the  West.  The  key  in  future  conflicts  thus  becomes  improved  combat 
effectiveness  and  thus,  ultimately,  the  development  of  improved  aircrew  selection  and 
training  programs. 

The  essence  of  these  programs  might  be  summarized  simply  as: 

1)  Using  an  effective  selection  method,  find  the  best  candidate  pilot, 

2)  Provide  him  her  with  the  best  and  most  cost  effective  training, 

3)  Significantly  improve  retention  rates  to  maximize  the  training  investment. 
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The  first  step,  selecting  the  best  pilot  candidate,  is  not  an  easy  one.  The  key- 
questions  become,  "What  mental  and  physical  characteristics  do  we  look  for  ?"  and 
"What  are  our  selection  criteria  ?"  The  answers  to  these  two  basic  questions  have  been 
the  object  of  much  of  the  pilot  candidate  selection  research,  which  will  be  addressed  in 
the  following  section. 

The  ROKAF  is  currently  experiencing  an  excessive  and  unacceptable  attrition 
rate  in  its  pilot  training  programs.  This  problem  is  further  compounded  by  the  fact 
that  the  total  number  of  candidates  admitted  to  the  flight  training  program  has  steadily- 
increased  since  the  early  1980's,  while  training  resources  remained  limited.  As  a  result 
the  candidates  who  remain  in  the  program  are  afforded  fewer  hours  of  instruction  in 
the  air  and  thus  do  not  receive  the  desired  quality  or  quantity  of  training. 

In  the  early  1960  s  both  the  U.S.  Navy  and  Air  Force  flight  training  programs 
suffered  similar  attrition  problems.  Figure  1.1  provides  a  summary  of  the  attrition 
categories  for  the  respective  services  at  that  time  and  Appendix  A  provides  a  detailed 
description  of  attrition  categories.  The  comparison  of  attrition  rates  between  USN  and 
IS\F  presented  in  Figure  1.1  as  a  result  of  various  factors.  L'SN  had  60.5%  attrition 
rate  and  21.5%  of  that  was  due  to  flight  failure.  Similarly  USAF  had  49.0%  due  to 
flight  failure  out  of  33.8%  attrition.  However,  with  the  significant  improvement  in 
existing  selection  methods  and  the  incorporation  of  new  methods  of  psychologicrl 
screening  both  services  were  able  to  decrease  their  respective  attrition  rates  by  almost 
30%  by  1977  [Ref.  2], 

The  present  research  attempts  to  summarize  some  of  these  improved  selection 
methods  utilized  by  both  the  L'.S.  Navy  and  Air  Force  and  presents  recommendations 
based  on  this  summary  for  improving  the  pilot  selection  program  of  the  ROKAF. 

B.  OBJECTIVE 

The  Korean  Air  Force  (ROKAF)  needs  a  more  effective  way  to  select  and  predict 
performance  of  pilot  candidates  for  success  in  primary  flight  training  and  operational 
flight  training  to  maintain  its  air  superiority.  The  purpose  of  this  thesis,  therefore,  is  to 
suggest  a  pilot  selection  system  which  selects  potential  successes  in  flight  training  and 
improves  pilot  quality.  This  research  also  provides  a)  an  overview  of  pilot  selection 
research,  b)  testing  methods,  and  c)  a  related  decision  theory  and  cost  effectiveness 
model. 
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Motivational  Attrition 


NOM  1S.4%  ;  DOR1  42.1°.)  1  SIE  22.4% 

(Net  Officer  (Drop  on  Request  ;  (Self 

Material)  or  \  oluntarv  1  Initiated 

Withdrawal)  Elimination) 


Flight 


FF  21.5% 

(Flight  Failure) 

! 

FD  49.0% 

(Flying  Deficiency) 

Medical 

j  NPQ  14.9% 

'  'Not  Phvsicallv 
Qualified) 

— 

M  12.3% 

(Medical) 

Academic 

!  2.0% 

2.9% 

_ 

Other 

|  1.1% 

2.0% 

'includes  3.6%  Air  Sickness. 


MOA  11.4% 

( Manifestation 
of  Apprehension) 


Figure  1.1  Representative  Navy  and  Air  Force  Pilot  Attrition  Categories  (From  Ref.2). 


II.  U.S.  PILOT  SELECTION  RESEARCH 


A.  WORLD  WARS 

It  was  first  noted  during  WW  I  that  a  significant  number  of  aircraft  accidents 
were  due  not  primarily  to  aircraft  failures  but  to  human  error.  Thus,  based  on  this 
data  France,  Italy  and  England  all  initiated  programs  to  aid  in  the  prediction  of  pilot 
success  during  primary  flight  training.  U.S.  efforts  began  as  early  as  1919.  [Ref.  3) 
Early  selection  tests  in  U.S.  were  based  primarily  on  Paper-and-Pencil 

Perceptual,  Cognitive  tests  with  psychomotor  devices  and  were  able  to  predict  flight 
performance  of  pilot  candidates  with  resonable  validity. 

The  development  of  selection  tests  continued  between  the  First  and  Second 
World  Wars.  Much  of  this  effort  was  aimed  at  the  development  of  psychomotor  test 
devices  which  simulated  stick  and  rudder  movements  of  the  aircraft. 

Perceptual  Cognitive  Paper-and-Pencil  tests  likewise  continued  to  be  developed  but 
both  these  and  the  psychomotor  tests  lacked  significant  predictive  validity.  A 
contributor  to  this  fact  was  the  deficiency  in  external  criteria  of  in-flight  performance. 
During  WW  II  the  demand  to  train  large  numbers  of  pilots  in  a  short  amount  of  time 
along  with  increased  training  cost  created  the  necessity  for  minimizing  attrition  due  to 
poor  proficiency.  The  Navy  thus  incorporated  in  its  selection  test  battery  numerous 
items  previously  evaluated  in  the  civilian  pilot  training  program.  In  the  "Pensacola 
1000  aviator  study",  evaluation  was  made  of  the  validity  of  more  than  60  psychological, 
physiological  and  psychomotor  test  items  from  the  Navy  selection  test  battery  utilizing 
over  900  Navy  flight  candidates.  Results  indicated  that  psychological  and  psychomotor 
measures  exhibited  greater  validity  for  the  prediction  of  success  in  flight  training  than 
did  physiological  measures  and  verified  the  validity  and  effectiveness  of  these 
psychological  instruments  and  psychomotor  tests  in  the  prediction  of  flight  success. 
[Ref.  3] 

The  US  Army  and  Air  Force  also  developed  their  pilot  selection  testing  battery 
during  WW  II  and  several  useful  methods  such  as  the  Complex  Coordination  test  and 
Divided  Attention  test  are  still  used  in  the  present  selection  test  battery.  The  Complex 
Coordination  test,  similar  to  USN  one,  used  by  the  requires  the  candidate  to  make 
simple  controlled  movements  of  a  stick  and  rudder  in  response  to  patterns  of  visual 
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stimuli.  During  WW  II  it  was  the  most  useful  and. most  highly  weighted  test.  Other 
testing  methods  developed  included  the  Finger  Dexterity,  Rudder  Control  Test  and 
Two  Hand  Pursuit  Test.  Typical  apparatus  test  validity  coefficients  for  pilots, 
navigators  and  bombardiers  are  presented  in  Table  1  [Ref.  4).  Note  that  the  test  most 
highly  related  to  pilots  is  the  Complex  Coordination  Test.  The  Discrimination 
Reaction  Time  is  significantly  related  to  navigators  and  bombardiers. 


TABLE  1 

VALIDITY  OF  APPARATUS  TESTS 


(year,  sample  size) 


Complex  Coordination 

Pilots 

Navigators 

Bombardiers 

i 

1 

I 

(1943,  3151) 
r*  .33,  rc  =  .40 

(1942,  1022) 
r  =  .17,  rc=.24 

(1943,  1829) 
r*  .10,  rc  =  .  1 3 

i  Rotarv  Pursuit  with 
:  Divided  Attention 

(1943,  3146) 
r  =  .14,  rc=  .22 

|  Finger  Dexterity 

! 

(1943,  4779) 
r-  .07,  rc„10 

(1943.  1828) 
r*  .13.  rc_15 

;  Discrimination 

1  Reaction  Time 

(1943,  4779) 
r-  .25,  rc».28 

(1942,  1829) 
r“ .22,  rc*.25 

!  Rudder  Control  Test 

(1943,  3146) 
r*  .22,  rc-.30 

Two  Hand  Pursuit 

(1943,1385) 
r*.27,  (average) 

i  Two  Hand 
!  Coordination 

(1943,4779) 
r— .31.  rc-.35 

(1942,  1022) 
rm .26,  rc-.29 

(1943.  1828) 
r-  09,  rc*.12 

r  -  validity  coefficient  based  on  dichotomous  pass;  fail  training  criterion. 
rc  =  validity  coefficient  corrected  for  restriction  in  range. 

( From  Reference  4) 
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B.  postwar  research 

Psychomotor  Tests  were  used  in  LSAF  selection  battery  until  1951  but  were 
deleted  due  to  administrative,  quality  control  and  reliability  problems.  A  subsequent 
study  of  the  revision  of  the  psychomotor  tests  indicated  that  the  use  of  simple  motor 
skill  tests  offered  little  if  any  predictive  value  for  flight  school  success.  It  was 
concluded  that  this  was  due  to:  [Ref.  5] 

1)  The  unreliability  of  both  predictor  and  criterion  scores. 

2)  The  use  of  inappropriate  motor  tests  for  the  particular  criteria  being 
evaluated,  and 

3)  The  task  of  flying  an  airplane  is  too  complex  for  simple  motor  skill  tests  to  be 
of  substantial  benefit. 

This  indicated  that  other  more  complex  abilities  were  necessary  for  successful 
pilot  performance  and  that  the  goal  of  future  research  should  be  to: 

1)  Define  the  abilities, 

2)  Quantify  the  basic  behaviors  making  up  the  abilities, 

3)  Devise  accurate  test  measures. 

These  problems  were  considered  continuously  through  the  1970s.  As  a  result, 
currently  the  L'SAF  has  been  achieving  great  success  in  predicting  flight  performance 
in  its  LPT  program.  This  success  will  be  addressed  in  more  detail  in  Chapter  III. 

The  other  type  test  which  has  been  the  most  widely  and  easily  used  as  a  selection 
test  has  been  the  Perceptual,  Cognitive  Paper-and-Pencil  tests.  The  L'.S.  military 
service  paper-and-pencil  test  battery  consisted  of :  [Ref.  3] 

1 )  A  general  intelligence  component  composed  of  verbal  and  quantitative  items, 

2)  Mechanical  comprehension, 

3)  A  spatial  component,  and 

4)  A  background  or  biographical  inventory  composed  of  miscellaneous  subset  of 
items  usually  of  an  historical  nature  known  to  relate  to  aviator  success. 

The  analysis  of  EEG  (Electroencephalogram)  recordings  has  also  received  much 
attention  as  a  possible  predictor  of  aircraft  accidents.  It  has  been  found  that 
individuals  with  abnormal  EEG's  are  involved  in  accidents  at  three  times  the  rate  of 
those  individuals  with  normal  EEG's1  [Ref.  3).  As  a  result  of  these  and  similar  findings 
the  Saw  now  screens  all  new  student  aviators  for  abnormal  EEG's  prior  to 
commencement  of  flight  training. 


'These  findings  have  resulted  in  the  Danish  Air  Force's  use  of  EEG  recordings  in 
the  selection  of  pilot  candidates. 
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C.  CURRENT  RESEARCH 

The  establishment  of  reliable  criterion  measures  is  a  prime  factor  in  any  selection 
process.  This  continues  to  be  an  extremely  difficult  and  dynamic  task.  For  example, 
the  Navy  has  recently  found  that  selection  variables  or  criterion  used  to  predict 
undergraduate  training  success  and  early  aviation  training  performance  measures  may 
not  be  related  or  may  even  be  inversely  related  to  performance  in  the  advanced 
operational  environment  [Ref.  3]. 

One  facility  which  has  been  invaluable  in  providing  insight  into  the  establishment 
of  viable  performance  criterion  for  possible  use  in  selection  tests  is  the  Navy  Air 
Combat  Maneuvering  Range  (ACMR).  This  facility  allows  multiple  fighter  aircraft  to 
engage  in  a  tactical  environment,  allowing  the  simulated  employment  of  air  to  air 
weapons  as  a  means  of  providing  training  in  tactical  skills,  weapon  utilization  and 
weapon  envelope  recognition.  Several  problems  arise  from  utilizing  ACMR  generated 
critenon,  however.  First,  the  high  psychological  stress  levels  associated  with  actual 
combat  may  be  absent,  secondly,  individual  performance  is  dependent  on  adversary 
performance.  Thus  an  independent  performance  assessment  is  difficult  if  not 
impossible.  Plans  are  ongoing  to  better  utilize  this  facility  in  understanding  aviator 
skills  which  appear  to  enhance  ACMR  performance. 

Finally,  Perceptual  Psychomotor  Testing  and  Division  of  Attention  Tests  are 
being  improved  by  both  the  L'.S.  and  Israeli  Air  Force  respectively.  Psychomotor 
testing  has  already  made  a  unique  contribution  to  the  L'SAF  UPT  selection  program. 
Israeli  Air  Force  has  likewise  experienced  success  in  predicting  training  success  in  high 
performance  jet  aircraft  using  the  Division  of  Attention  Test2  [Ref.  6]. 

We  have  now  discussed  what  testing  methods  have  been  developed  and  exist 
today,  and  have  established  how  important  they  are  in  developing  the  pilot  selection 
program.  We  now  suggest  useful  pilot  selection  methods  for  the  ROKAF  which 
hopefully  will  result  in  considerable  resource  savings  and  significant  improvements  in 
pilot  quality. 


Approximately  for  2,000  subjects,  the  data  was  significantly  related  to  success  in 
jet  training. 
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III.  AVIATOR  SELECTION  METHODS  FOR  ROKAF 


Although  the  human  resources  and  materials  for  the  flight  training  program  in 
the  ROKAF  have  increased  only  slightly,  the  number  of  pilot  candidates  is  increasing 
every  year  as  the  number  of  Korean  Air  Force  Academy  (KAFA)  cadets  increases. 
Since  the  only  admission  requirement  to  the  KAFA  for  flight  training  is  a  medical 
clearance,  and  most  pass  the  medical  examination,  the  ROKAF  pilot  training 
command  cannot  handle  the  number  of  cadets  and  candidates  from  other  sources 
(AFROTC  and  Navy).  This  increased  number  of  pilot  candidates  has  created  the 
necessity  for  minimizing  the  number  of  candidates  lost  due  to  poor  or  unsatisfactory 
proficiency  in  the  UPT  program;  maximizing  the  probability  that  potential  failures  will 
be  screened  out  prior  to  flight  training. 

The  present  pilot  selection  procedures  of  the  ROKAF  do  not  adequately  take 
into  account  candidates'  mental  abilities.  The  electrocadiogram,  however,  is  being  used 
for  fighter  pilots,  and  an  abnormal  electrocadiogram  will  prohibit  continuation  of  flight 
training.  All  of  the  KAFA  cadets  who  pass  the  medical  examination  enter  the  flight 
training  program  and  have  a  chance  to  fly.  Flight  training  starts  early  before 
graduation  from  the  KAFA  and  ends  about  a  year  after  graduation.  This  system,  with 
flight  training  begining  early  before  graduation  was  adapted  during  the  1980's. 
However  the  current  system  does  not  appear  to  represent  a  significant  improvement  in 
previous  program.  Cadets  who  fail  as  a  cadet  find  it  difficult  to  continue  the  program 
and  those  who  fail  as  an  oflicer  after  graduation  and  commissioning  And  it  difficult  to 
adapt  to  different  specialties.  Candidates  are  divided  into  several  subgroups  in  order  to 
meet  the  limited  training  squadron  resources.  Graduates  of  the  subgroups  enter  the 
next  phase  of  training  with  attrites  returning  to  the  KAFA.  Attritions  continue 
through  the  flight  training  -  from  the  primary  training  program  up  to  the  advanced 
program  over  several  years.  The  only  way  that  candidates  could  be  differentiated  is  on 
some  amount  of  actual  flying  at  the  training  squadron  and  the  failures  would  be  totally 
subjective  based  on  instructors'  feeling,  knowledge  and  their  experiences,  except  for 
voluntary  withdrawal  or  drop  on  request  (DOR). 

The  following  demonstrates  a  useful  way  to  help  in  solving  the  existing  short 
comings  of  the  current  ROKAF  pilot  training  program  and  suggests  pilot  candidate 
selection  methods  which  might  be  used  as  a  starting  point  in  alleviating  these  problems. 


A.  METHODS 

This  section  suggests  selection  methods  for  the  ROKAF  which  are  currently 
being  used  in  the  U.S.  The  purpose  of  the  tests  are  to  select  candidates  who  have  a 
high  probability  of  completing  the  flight  training  program.  Multitask  performance  is 
being  used  in  L'S\  and  it  has  enjoyed  considerable  success  in  predicting  flight  training 
(Ref.  7).  The  USAF  uses  a  psychomotor  screening  system  for  selecting  candidates  as 
well  as  identifying  whether  the  candidate  is  suited  for  fighter  pilot  training  [Ref.  8]. 
The  USAF  also  suggests  that  an  integrated  model  including  psychomotor  testing  and 
other  available  information  -  AFOQT  scores,  technical  degree,  prior  flying  experience, 
age  of  subjects,  etc.  -  is  better  than  using  only  the  psychomotor  test.  All  three 
methods  use  a  psychomotor  test  since  the  psychomotor  scores  have  been  found  to 
relate  significantly  to  flight  training  pass;  fail  criteria.  As  the  USN  adapted  the  EEG 
recordings  as  a  direct  result  of  the  Danish  Air  Force  as  we  discussed  in  Chapter  II,  the 
ROKAF  may  choose  one  of  these  three  methods  or  a  combination  and  apply  it  to 
ROKAF  cadet  selection  with  some  possible  modifications.  The  following  text  describes 
the  three  methods  which  could  be  used  and  modified  for  the  ROKAF  and  shows  how 
the  predicted  results  may  be  related  to  the  actual  outcomes. 

1.  Multitask  Performance 

The  USN  had  great  success  in  predicting  student  success  in  primary  flight 
training  utilizing  multitask  performance  tests  in  1986  [Ref.  7],  A  Psychomotor  Test 
(PMT)  and  a  Dichotic  Listening  Task  (DLT)  were  administered  under  single-task  and 
multitask  conditions  separately  and  in  combination.  Testing  procedures  were  as 
follows:  [Ref.  7] 

1)  Psychomotor  Test  (PMT) 

•  The  PMT  requires  subjects  to  simultaneously  center  two  visually 
presented  cursors  on  fixed  targets  displayed  on  a  CRT  screen.  The  test 
performance  measurements  were  the  accumulated  X,  Y,  and  Z  axis  error 
distances  between  the  target  and  the  cursor  point.  A  diagram  of  the 
experimental  apparatus  of  Psychomotor  Task  (PMT)  is  depicted  in  Figure 
3.1  . 

2)  Dichotic  Listening  Task  (DLT) 

•  The  DLT  first  requires  subjects  to  attend  to  auditory  information 
presented  to  one  ear,  while  ignoring  similair  information  presented  to  the 
opposite  ear,  and  then  after  considering  an  auditory  cue,  to  switch 
attention  rapidly  to  the  previously  attended  ear,  or  maintain  attention  to 
the  previously  attended  channel.  The  modified  Dichotic  Listening  task 
(DLT)  [Ref.  9[  consisted  of  letter-digit  string  sets  of  24  listening  trials  for 
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Figure  3.1  Experimental  Apparatus  (adapted  from  Ref.7). 


each  of  3  counter-balanced  response  modalities.  Written,  keypad  and 
verbal  response  conditions  were  employed.  The  subjects  were  instructed 
to  direct  attention  to  one  ear  while  ignoring  the  other,  and  correctly 
report  the  nine  digits  per  trial.  An  example  DLT  trial  is  depicted  in 
Figure  3.2  .  The  single-task  performance  measure  for  the  DLT  was 
simply  the  number  of  correct  responses  per  24  trials. 

3)  Multitask  Performance 

•  For  the  multitask  condition,  subjects  performed  two  sessions  of  the  DLT 
and  PMT  simultaneously.  Performance  measures  for  the  PMT  and  DLT 
in  the  multitask  conditions  were  identical  to  those  of  the  single-task 
conditions  except  that  multitask  DLT  performance  was  based  on  12 
rather  than  24  trials. 

The  tests  were  performed  in  order  such  as: 

1)  Psychomotor  Test  (PMT), 

2)  Dichotic  Listening  Task  (DLT)  and 
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PARTI 


Left  Ear  R8NSM  Y2GB7FL6RL5 

'Right"  (Vocal  Channel  "attend"  Command) 

Right  Ear  YL3SR4FZ9XFOFN1L 

PART  II 

Left  Ear  B  F  4  3  7  9 

“Left"  (Vocal  Channel  "attend"  Command) 

Right  Ear  G  L  1  5  6  2 


i 

1 


Figure  3.2  DLT  Trial  Examples  (adapted  from  Ref.7). 

3)  Multitask,  that  is,  PMT  and  DLT  simultaneously. 

To  validate  the  test  criteria,  subjects'  flight  performance  were  tracked  until 
completing  pilot  training.  Table  2  shows  the  mean  and  pass,  fail  correlation  of  each 
test  measure.  Correlations  between  each  testing  measure  are  presented  in  Table  3  .  As 
you  can  see  in  Table  2  ,  testing  measures  1,  5,  9,  and  11  are  significantly  related  to  the 
flight  training  pass  fail  criteria.  Table  3  presents  a  correlation  matrix  of  test  measures. 

2.  Psychomotor  Test 

The  L'SAF  has  utilized  psychomotor  testing  in  relation  to  pilot  selection.  The 
LSAF  primary  objective  has  been  an  attempt  to  develop  reliable  and  valid  test  devices 
and  validation  of  test  scores.  In  1969,  the  Air  Force  Human  Resource  Laboratory 
(AFHRL)  attempted  to  determine  the  reliability  of  two  computer-based  psychomotor 
tests  (i.e.,  Two-Hand  Coordination  and  Complex  Coordination)  [Ref.  8).  Both  tests 
were  employed  to  measure  the  subject's  ability  to  coordinate  movements.  Subjects 
were  from  Air  Force  Academy  (AFA),  Air  Force  Reserve  Officer  Training  Corps 
(AFROTC),  and  Officer  Training  School  (OTS).  The  largest  group  of  the  sample  was 
AFROTC  and  additional  subjects  from  Williams  AFR  were  used  for  validation  of 
testing  results. 
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TABLE  2 

DESCRIPTIVE  STATISTICS  FOR  SINGLE  AND  MULTITASK 
MEASURES  AND  SELECTION  TESTS 


(percent  correct) 


Test  Measure 

Mean 

Pass;  Fail 
(r) 

1.  PMT  Session  1,  X,Y,Z  error  combined 

19116.36 

-.283 

2.  PMT  Session  2,  X,Y,Z  error  combined 

11541.20 

-.028 

3.  PMT  Session  3,  X,Y,Z  error  combined 

9896.90 

-.241 

4.  PMT  Session  4,  X,Y,Z  error  combined 

9550.82 

-.120 

5.  DLT  written  response,  No.  correct 

211.48  (97.9%) 

.360 

6.  DLT  keypad  response,  No.  correct 

210.60  (97.5%) 

.162 

7.  DLT  vocal  response,  No.  correct 

207.52  (96.1%) 

.244 

8.  PMT  X,Y,Z  error  (with  keypad  DLT) 

21816.92 

.036 

9.  DLT  keypad  response  (with  PMT) 

102.34  (94.8%) 

.413 

10.  PMT  X,Y,Z  error  (with  vocal  DLT) 

14038.10 

-.002 

11.  DLT  vocal  response  (with  PMT) 

101.44  (93.9%) 

.395 

12.  AQT  standard  score 

5.66 

.172 

13.  FAR  standard  score 

6.80 

.361 

**  Measure  1-7  :  single- task 

**  Measure  8-11  :  multitask 

**  Measure  12,13  :  selection  test 

(From  Reference  7) 

Testing  procedures  were  as  follows:  [Ref.  8] 

1)  Two-Hand  Coordination 

•  The  first  of  the  tests  presented  a  triangular-shaped  target  and  a  cross¬ 
shaped  pipper  on  the  CRT  (Figure  3.3).  The  computer  moved  the  target 
in  an  elliptical  path  and  with  varying  speeds  -  faster  near  the  4  o'clock 
and  slower  near  the  1 1  o'clock  position.  The,  subjects  were  instructed  to 
use  both  joysticks  simultaneously  (one  is  for  up-down  and  the  other  is  for 
left-right)  in  a  coordinated  manner  to  move  the  pipper,  keeping  it  as  close 
as  possible  to  the  target  on  the  CRT.  The  computer  measured  absolute 
distance  from  the  pipper  to  the  target.  The  horizontal  error  (Xj)  and 
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TABLE  3 

CORRELATION  MATRIX  FOR  SINGLE  AND  MULTITASK 
MEASURES  AND  SELECTION  TESTS 

Measure  123456789  10  11  12 

1 

2  .401 

3  .727  .544 

4  .520  .667  .838 

5  -.626  -.131  -.694  -.443 

6  -.616  -.490  -.523  -.453  .540 

7  -.494  -.182  -.407  -.325  .708  .544 

8  .393  .500  .501  .597  -.385  -.503  -.612 

9  -.708  -.244  -.782  -.508  .869  .588  .563  -.348 

10  .526  .298  .532  .443  -.531  -.468  -.669  .873  -.445 

11  -.737  -.243  -.732  -.457  .913  .540  .644  -363  .887  -.529 

12  -.157  -067  .060  -021  .058  .104  .147  .048  .116  .037  .058 

13  -.368  -.239  -.223  -.287  .050  .144  .223  .254  .144  .230  .099  .511 

**  See  Table  2  for  description  of  numbered  variables. 

(From  Reference  7) 

vertical  error  (Yj)  measurement  were  accumulated  during  5-min  test 
period. 

2)  Complex  Coordination 

•  The  second  test  presented  a  set  of  cross-hairs  centered  on  the  CRT,  a  dot¬ 
shaped  pipper,  and  a  thin  vertical  bar  at  the  bottom  of  the  CRT  (Figure 
3.4).  The  subject  controlled  the  pipper,  both  horizontally  and  vertically, 
using  the  floor-mounted  joystick.  The  control  responses  were  the  reverse 
of  what  is  traditionally  required  on  aircraft  to  reduce  the  advantage  of  a 
subject  with  prior  flying  experience.  The  subjects  were  instructed  to  use 
the  joystick  to  center  the  pipper  horizontally  and  vertically  on  the 
intersection  of  the  cross-hairs  and  simultaneously  press  the  appropriate 
rudder-style  pedal  to  center  the  "rudder  bar"  over  the  lower  part  of  the 
vertical  cross-hair.  The  computer  measured  in  inches  and  accumulated 
during  5-min  test  period  the  the  absolute  distance  from  the  pipper  to  the 
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PSYCHOMOTOR  TEST  I 


LEFT  —  RIGHT  HAND  COORDINATION 
(10  minutes) 


Figure  3.3  Two-Hand  Coordination  Test  Depiction  (adapted  from  R.ef.7). 

intersection  of  the  cross-hairs  and  from  the  rudder  bar  to  the  vertical 
cross-hair.  The  three  scores  obtained  from  this  test  were  the  cumulative 
horizontal  error  (X2),  vertical  error  (Y2),  and  horizontal  error  for  rudder 
bar  (Z2). 

Table  4  presents  five  error  scores  (Xj,  Yj,  X2.  Y2,  Z2)  for  each  subject. 
These  data  show  that  the  Two-Hand  Coordination  test  was  more  difficult  than  the 
Complex  Coordination.  There  might  also  be  some  common  information  about  the 
subjects  since  all  five  test  scores  measured  psychomotor  ability.  Table  5  shows 
intercorrelations  across  the  tests.  Two  psychomotor  tests  were  measuring  different 
abilities  with  very  little  in  common  between  the  tests  as  indicated  in  Table  5  .  To 
validate  the  psychomotor  testing  scores,  the  mean  scores  of  UPT  graduates  and 
eliminees  were  compared  following  completion  of  flight  training.  Table  6  presents  the 
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PSYCHOMOTOR  TEST  II 


STICK  &  RUDDER  SKILLS 
(10  minutes) 


PIPPER 

RUDDER  BAR 


ar 


Figure  3.4  Complex  Coordination  Test  Depiction  (adapted  from  Ref.7). 

mean  of  the  psychomotor  scores  among  three  categories:  UPT  graduates,  all  UPT 
eliminees,  and  UPT  flying  deficiency  eliminees.  All  five  scores  had  significant 
differences  between  graduates  and  either  category  of  eliminees.  Note  that  for  ail  three 
of  the  Complex  scores  (X2,  Y2.  and  Zi)  were  significantly  worse  than  the  graduates' 
means.  These  results  were  used  to  identify  subjects  more  likely  to  be  fighter  pilots. 
Figure  3.5  presents  this  result  of  all  five  means  for  both  fighter- recommended  (FR) 
and  non-FR  students.  The  FR  students  had  significantly  better  scores  than  the  non- 
FR  students.  The  psychomotor  screening  equation  was  finally  developed  and  the  most 
useful  screening  score  based  on  psychomotor  ability  can  be  computed  from  the 
following  equation  [Ref.  8): 
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TABLE  4 

DESCRIPTIVE  STATISTICS 


(inch) 


1  Test  Score 

Mean 

SD 

Range  j 

|  Two-Hand  Coordination 

j 

i  XI  (Horizon) 

14,709 

5,400 

428  -  65,268 

YI  (Vertical) 

16,748 

5,538 

5,687  -  57,292 

Complex  Coordination 

X2  (Horizon) 

4,514 

6,520 

326  -  71.040 

Y2  (Vertical) 

3,860 

6,445 

486  -  76,320 

Z2  (Rudder) 

5,580 

6,018 

167  -  71.040 

(From  Reference  8) 


TABLE  5 

CORRELATION  MATRIX 


X1 

Yl 

X, 

Y2 

XI  (Horizon) 

Yl  (Vertical) 

.87 

X2  (Horizon) 

.20 

.25 

Y2  (Vertical) 

.19 

.23 

.86 

Z2  ( Rudder) 

.20 

.24 

.77 

.71 

(From  Reference  8) 


Yj  *  a  +  (bj  x  Xlj)  +  (b2  x  Y2j) 

where  Yj  =  psychomotor  screening  score  of  ith  subject 
a  =  constant  (0.95648) 
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TABLE  6 

MEAN  PSYCHOMOTOR  SCORES  BY  LPT  OUTCOME 


X1 

14,315 

Y1 

16,341 

X2 

y2 

Z2 

Graduate  Means  (X  =  1,348) 

3.559 

2,058 

4.725 

All  Elim.  Means  (X  =  377) 

15.829 

17,621 

4.936 

4,173 

6,678 

Flying  Deficiency  (FD) 

16,302 

18.007 

5,593 

4.702 

~.5S0 

Elim.  Means  (N  =  199) 

(From  Reference  8) 

!  i 

bj  =  weight  of  psychomotor  XI  score  (-.0000075748) 
b,  =  weight  of  psychomotor  Y2  score  (-.000020375) 

Xlj  =  psychomotor  Xj  score  of  <th  subject 
Y2|  =  psychomotor  Y->  score  of  ith  subject 

This  equation  provides  a  screening  score  based  on  psvchomotor  ability.  These  scores 
could  be  used  in  a  selection  system  by  rank  ordering;  and  the  best  candidates  could  be 
selected  to  fill  the  required  training  quotas. 

5.  Integrated  Selection  System 

Although  psychomotor  testing  scores  are  significantly  related  to  pass  fail 
criteria  in  LPT,  the  ideal  pilot  selection  method  would  consider  all  available 
informations  on  the  subjects. 

Current  operational  USAF  pilot  selection  criteria  consider  information  from 
the  five  composite  scores  of  the  AFOQT,  age  at  the  start  of  UPT,  possession  of  a 
college  technical  degree,  and  possession  of  a  private  pilot  s  license  or  completion  of  a 
USAF  light  aircraft  flying  program  [Ref.  8).  Table  7  lists  the  correlations  of  all 
available  measures  with  the  UPT  pass ' Fail  criterion  for  all  reasons  (FY79  to  FY83). 

The  integrated  model  was  developed  based  on  the  USAF  psychomotor  testing. 
Because  pilot  candidates  are  chosen  differently,  depending  on  their  source  of 
commission,  three  different  Integrated  Pilot  Candidate  Selection  Models  (IPCSMs) 
were  developed. 

The  three  IPCSMs  equations  were  developed  and  will  produce  a  score 
corresponding  to  the  probability  of  UPT  success  using  the  following  equations  : 
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TABLE  7 

CORRELATIONS  OF  IPCSM  PREDICTIONS  WITH  LPT  OUTCOME 


Predictors  Correlation  with  LPT 

Pass  Fail 

Sample  Si. 

FSP  Pass  Fail 

.137 

1,534 

FSP  Final  Grade 

.271 

1,534 

FSP  Sum  of  Grades 

.303 

1,534 

AFOQT  Pilot  Score 

.158 

4,460 

AFOQT  Navigator  Tech  Score 

.148 

4,460 

AFOQT  Academic  Score 

.080 

4,577 

AFOQT  Verbal  Score 

.007 

4,576 

AFOQT  Quantitative  Score 

.138 

4,577 

Psychomotor  XI  Score 

-.118 

1,918 

Psychomotor  Y1  Score 

-.099 

1,919 

Psychomotor  X2  Score 

-.153 

1,906 

Psychomotor  Y2  Score 

-.181 

1,910 

Psychomotor  Z2  Score 

-.146 

1,916 

Age 

-.120 

8.438 

Gender 

.033 

8,438 

Race 

.110 

8,292 

Possession  of  Tech  Degree 

.111 

8,183 

(From  Reference  8) 


1)  IPCSM  I  (OTS  Model) 


Yj  -  a  +  (b,  x  Xlj)  +  (b2  x  Y2j)  +  (b3  x  agej) 
+  (b4  x  AFOQTPj)  +  (b5  x  AFOQTNj) 
+  (b6  x  FSPij)  +  (b7  x  FSP2j) 


where  Yj 
a 


b 


1 


*  IPCSM  I  score  for  ith  subject 
=  constant  (0.10192) 

*  weight  for  psychomotor  XI  score  (*.000015211) 
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b2  *  weight  for  psychomotor  Y2  score  (-.000010618) 

bj  =  weight  for  age  of  subject  (-.0012) 

b4  =  weight  for  AFOQTP/AFOQT  pilot  score  (0.00112) 

bs  =  weight  for  AFOQTN/AFOQT  navigator  score  (0.00029067) 

b6  =  weight  for  FSP1/FSP  final  grade  (0.08249) 

b7  =  weight  for  FSP2/FSP  sum  of  grades  (0.01923) 

XI  j  =  psychomotor  XI  score  of  /th  subject 

=  psychomotor  Y2  score  of  /th  subject 
agej  =  age  at  start  of  LPT  of/th  subject 

AFOQTP-  =  AFOQT  pilot  composite  score  of  /th  subject 
AFOQTNj  =  AFOQT  navigator  composite  score  of  /th  subject 
FSPlj  =  FSP  final  grade  of /th  subject 
FSP2j  =  FSP  sum  of  grade  of  /th  subject 


2)  IPCSM  II  (AFROTC  Model) 

Yj  =  a  +  (^  x  Xlj)  +  (b2  x  Y2j)  +  (b3  x  agej) 

+  (b4  x  AFOQTPj)  +  (b$  x  AFOQTNj) 

where  Yj  =*  IPCSM  II  score  for /th  subject 
a  =  constant  (1.72485) 

bj  =  weight  for  psychomotor  XI  score  (-.0000030409) 

b2  =»  weight  for  psychomotor  Y2  score  (-.000022526) 

b,  =  weight  for  age  of  subject  (-.04517) 

b4  =  weight  for  AFOQTP/AFOQT  pilot  score  (0.00124) 

b5  =  weight  for  AFOQTN/AFOQT  navigator  score  (0.00225) 


3)  IPCSM  III  (AFA  Model) 

Yj  =  a  +  (b,  x  xij)  +  (b2  x  Y2j)  +  (b3  x  agej)  +  (b4  x  techj) 

where  Yj  m  IPCSM  III  score  of  /th  subject 
a  »  constant  (1.43056) 

bj  *  weight  for  psychomotor  XI  score  (-.0000050818) 

b2  *  weight  for  psychomotor  Y2  score  (-.0000072309) 

b3  *  weight  for  age  of  subject  (-.021 11) 

b4  *  weight  for  possession  of  technica’  degree  (.08756) 

techj  =  1  if  /th  subject  has  a  technical  degree,  0  otherwise 
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The  mean  of  the  IPCSM  scores  are  graphically  presented  in  Figure  3.6  .  The 
differences  between  the  means  were  significant  and  generated  the  better  prediction  than 
using  the  psychomotor  test  only.  One  way  to  select  candidates  is  to  accept  only  those 
candidates  who  obtain  a  screening  score  above  a  predetermined  cutoff.  The  cutoff 
score  would  be  determined  by  the  quota  required  and  attrition  rate.  The  higher  the 
cutoff  score  is  set,  the  greater  the  impact  on  attrition  in  pilot  training  program.  The 
FY83  LPT  sample  was  used  to  cross-vaiidate  the  three  IPCSMs  and  resulted  in  the 
IPCSM  scores  were  significantly  correlated  with  LPT  outcome.  Since  the  "Whole 
Person  Concept"  is  imbedded  in  this  multi-dimensional  IPCSM  approach  to  pilot 
selection,  the  integrated  selection  method  has  better  predictors  than  the  psychomotor 
test  only  does. 

As  we  can  see  through  the  Chapter,  all  three  pilot  selection  methods  include  a 
psychomotor  testing  which  means  psychomotor  testing  scores  are  significantly  different 
between  the  successes  and  the  failures.  It  should  have  the  same  result  if  psychomotor 
tests  were  applied  to  ROKAF.  The  only  thing  different  between  L.S.  and  ROKAF 
would  be  the  cutoff  score  as  a  different  required  quota.  The  ROKAF  decision  maker, 
however,  must  know  the  advantages  of  using  an  experimental  testing  method  in  terms 
of  their  utilities  or  cost  effectiveness.  Chapter  IV  provides  one  way  to  make  a  decision 
and  evaluate  the  outcomes. 


IV.  EVALUATION  OF  TESTING  METHOD 


This  chapter  discusses  how  to  make  appropriate  decisions  which  maximize  an 
overall  Measure  of  Effectiveness  (MOE)  or  Utility  which  might  be  used  in  a  pilot 
selection  program.  The  final  goal  of  personnel  selection  programs  has  traditionally 
been  to  achieve  maximum  accuracy  of  measurement  and  efficiency  of  prediction. 
Decision  theory'  recognizes  that  the  outcomes  of  predictions  are  of  primary  importance 
to  individuals  and  organizations.  As  discussed  in  Chapter  III,  for  example,  Korean 
AFA  cadets  are  commissioned  as  officers  after  four  years  of  attendance.  Some  fail 
either  in  undergraduate  or  operational  flight  training.  This  results  in  high  attrition 
cost,  dissatisfaction,  lowered  morale,  and  a  number  of  officers  totally  unneeded  in  the 
service.  Since  the  measurement  and  prediction  are  simply  technical  components  of  a 
system  designed  to  make  decisions,  we  have  to  define  the  MOE  before  making  any 
decisions.  There  are,  however,  a  number  of  possible  MOEs  in  most  decision  problems. 
For  example,  there  would  be  three  possible  MOEs  in  a  pilot  selection  problem: 

1)  Costs  -  selection,  attrition  and  training, 

2)  Pilot  quality,  and 

3)  Probability  that  the  quota  will  be  satisfied. 

Thus  decision  outcomes  should  be  evaluated  in  terms  of  their  consequences  for 
individuals  and  organizations.  That  is,  in  terms  of  their  overall  MOE  or  Utility.  This 
provides  us  with  a  consistent  way  in  which  to  combine  different  figures  of  merit  or 
subjective  and  judgemental  factors,  and  probabilistic  aspects  of  outcomes  into  an 
overall  MOE. 

To  construct  and  evaluate  a  decision  model,  we  have  to  find  parameters  which 
function  in  pilot  selection  outcomes.  The  major  factors  are  [Ref.  10): 

a)  the  base  rate  (the  proportion  of  persons  currently  accepted  who  are 
satisfactory) 

b)  the  selection  ratio  (the  proportion  of  applicants  accepted)  and 

c)  the  validity  coefficient  (the  predictor  criterion  correlation). 

Taylor  and  Russell  [Ref.  10|  published  a  series  of  tables  which  illustrated  that  the 
value  of  a  test  was  a  function  of  the  three  considerations  listed  above. 


31 


A.  BASE  RATE  (BR) 


t' 


Figure  4. 1  Four  possible  outcomes  of  personnel  selection  (adapted  from  Ref.  1 1 ). 

The  possible  outcomes  of  personnel  selection  programs  are  illustrated  in  Figure 
4.1  .  This  figure  depicts  a  bivariate  plot  for  a  selection  test  with  a  validity  of  .50.  The 
horizontal  axis,  BB',  represents  a  particular  base  rate  of  satisfactory  criterion 
performance,  in  this  case,  50  %.  The  vertical  axis,  CC',  denotes  the  cutting  score  for  a 
50  %  selection  ratio.  Thus,  Quadrant  A  contains  a  number  of  satisfactory  subjects 
above  the  cutting  score  (valid  positives,  VP),  Quadrant  B  contains  unsatisfactory 
subjects  above  the  cutting  score  (false  positives,  FP),  Quadrant  C  contains 
unsatisfactory  subjects  below  the  cutting  score  (valid  negatives,  VN),  and  Quadrant  D 
contains  satisfactory  subjects  below  the  cutting  score  (false  negatives,  FN).  (Ref.  11] 
Once  the  cutoff  score  is  set,  a  definite  number  of  predictions  of  success  and  failure  are 
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made.  As  Figure  4.1  shows,  the  total  number  of. predictive  positives.  Npp,  may  be 
calculated  from  : 


Npp  -  VP  +  FP 


Similarly,  the  total  number  of  predictive  negatives,  Npn,  is  given  by: 


Npn  =  FN  +  VN 


The  total  number  of  actual  positives,  N'a  ,  is  given  by: 


Nap  -  FN  +  VP 


And  the  total  number  of  actual  negatives,  Nan,  is  calculated  by 


Nan  "  VN  +  FP 


The  probability  of  a  valid  positives,  P(VP),  is  thus: 


P(  VP)  -  VP  (\ap  +  Nan)  =  VP- TN 

where  TN  is  the  total  number  of  candidates.  Figure  4.2  presents  the  probabilities  and 
their  relationships. 

When  outcomes  are  expressed  as  probabilities  they  are  referred  to  as  hit  rates  '. 
The  probabilities  or  proportions  of  actual  positives  that  exist  in  the  total  sample  of 
candidates  is  called  the  "Base  Rate"  (BR).3  The  probability  may  be  calculated  directly 
BR  =»  N  TN  or  from  the  probabilities  of  two  outcomes  BR  =  P(FN)  +  P(VP)  as  in 
Figure  4.2  .  The  BR  has  reference  to  the  base  rate  of  the  positive  class  and  so  the 
probability  or  base  rate  of  negative  in  the  total  sample  is  given  :  1  -  BR. 

Selection  measures  are  most  useful  when  BRs  are  about  .50  [Ref.  10].  As  the 
BR  departs  radically  in  either  direction  from  .50,  the  benefit  of  an  additional  predictor 
becomes  questionable. 


3The  proportion  of  candidates  who  would  be  successful  in  flight  training  without 
use  of  the  test. 


33 


Actual  Diagnosis 

Test  Diagnosis 

| 

Total  i 

Positive 

Negative 

1 

i 

P(VP)  =  VP  TN 

P(FN)=  FN  TN 

- 1 

BR  =  Nap  TN 

1 

valid  positive  rate 

false  negative  rate 

Base  Rate 

Positive 

(Proportion  of 

(Proportion  of 

(Probability  of 

positives  called 

positives  called 

actual  positives) 

positive) 

negative) 

P(FP)  =  FP/TN 

P(VN)=  VN  TN 

1  -  BR  = 

j 

false  positive  rate 

valid  negative  rate 

N  TN 

Negative 

(Proportion  of 

(Proportion  of 

(Probability  of 

i 

negatives  called 

negatives  called 

actual  negatives) 

positive) 

negative) 

SR-Npp.(Npp  +  N 

pn>  1  -  SR  - 

Selection  Ratio 

•^pn^  •''pp  + 'spn') 

Total 

(Probability  of 

(Probability  of 

predicted  positives) 

predicted  negatives) 

_ 

Figure  4.2  Outcomes  Expressed  as  Probabilities. 

B.  SELECTION  RATIO  (SR) 

The  probability  or  proportion  of  predicted  positives  among  the  total  sample  of 
candidates  is  called  the  'Selection  Ratio"  (SR).4  This  probability  may  be  calculated 
directly  as  follows: 


SR 


Npp  (Npp  +  Nptj) 


4The  proportion  of  candidates  selected  with  the  use  of  test. 


34 


or  from  the  probabilities  of  two  outcomes  such  that  - 


SR  =  P(VP)  +  P(FP) 

Whenever  a  quota  exists  on  the  total  number  of  candidates  that  may  be 
accepted,  the  selection  rate  becomes  more  important.  As  the  SR  approaches  1.0,  for 
example,  which  means  all  candidates  must  be  selected,  it  will  be  dominated  by  other 
alternatives  and  as  the  SR  approaches  zero,  it  dominates  all  other  alternatives  which 
means  a  decision  maker  can  afford  to  be  selective.  Cronbach  and  Glesser  [Ref.  12) 
have  shown  that  when  the  frequency  distributions  of  the  two  groups  (accept  reject)  are 
plotted  separately  along  the  same  base  line,  the  optimum  cutting  score  for 
distinguishing  between  the  two  groups  will  occur  at  that  point  where  the  two 
distributions  intersect.  However,  in  most  practical  selection  situations,  the  decision 
maker  is  not  free  to  vary'  selection  ratios. 

C.  VALIDITY  COEFFICIENT 

Evaluating  the  benefit  obtained  from  tests  is  of  considerable  practical  interest  and 
has  been  a  problem  in  industrial  psychology.  The  relation  of  benefit  to  validity  has 
long  been  regarded  as  an  important  question.  Most  attempts  to  evaluate  benefits  have 
focused  on  the  validity  coefficient.  As  at  least  five  formulas  for  interpreting  the  validity 
coefficient  have  been  given,  we  face  the  problem  of  choosing  between  them  or  of 
modifying  them.  [Ref.  12] 

The  validity  coefficient  is  usually  defined  as  the  correlation  of  test  score  with 
outcome  or  criterion  score.  We  shall  use  rxy  in  our  discussion  because  the  decision 
maker  is  interested  in  predicting  the  evaluated  outcome,  that  is,  the  outcome  expressed 
in  utility  units.  The  rule  having  the  longest  history  translates  the  validity  coefficient 
rxv  into  the  "index  of  forecasting  efficiency"  [Ref.  1 2], 


E  =  J  1  -  (1  -  r2xy)  =  |  rxy  | 

Although  the  validity  coefficient  rxy  is  a  useful  index,  there  are  many  ways  in 
which  it  falls  short  of  accurately  predicting  success.  From  a  practical  point  of  view,  the 
number  of  correct  decisions  made  by  a  psychological  test  or  assessment  is  more 
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important  than  the  degree  of  association  which  exists  between  predicted  and  obtained 
scores  [Ref.  12].  When  two  variables  are  expressed  in  dichotomous  form  (success- 
failure,  select-reject)  as  in  Figure  4.1  ,  the  correlation  between  them  is  given  by  the 
point-biserial  "rp  coefficient  '.  A  simplified  computational  formula  for  the  rp  coefficient 
is:  [Ref.  13] 


P(VP)  -  BR  x  SR 

TP  [  BR(1  -  BR)SR(1  -  SR)  ]  12 

The  value  of  rp  is  computed  knowing  only  three  values: 

1)  The  probability  of  a  valid  positive,  P(VP), 

2)  The  base  rate,  BR,  and 

3)  The  selection  ratio,  SR. 

The  computational  formula  for  the  rp  coefficient  can  be  solved  for  the  probability 
of  valid  positives: 


P(VP)  -  BR  x  SR  +  rpV  BR(1  -  BR)SR(1  -  SR) 

This  expression  is  especially  useful  in  evaluating  the  outcomes  of  test  prediction. 
Given  the  validity  of  a  test  r  ,  the  base  rate,  BR,  and  the  proportion  of  candidates  to 
be  selected,  SR.  it  is  possible  to  completely  specify  the  outcomes  of  predictions. 

Taylor  and  Russell  [Ref.  14]  denounced  the  practice  of  evaluating  the 
effectiveness  of  predictor  variables  solely  on  the  basis  of  their  validity  coefficients. 
They  prepared  tables  for  determining  the  effectiveness  of  a  selection  test  by  estimating 
the  proportion  of  selectees  that  are  considered  satisfactory  when  the  test  validity,  base 
rate,  and  selection  ratio  are  known  or  assumed. 

D.  DECISION  STRATEGIES 

There  are  mainly  two  different  decision  strategies  which  are  single-stage 
(nonsequential)  and  multi-stage  (sequential)  [Ref.  12].  The  major  difference  between 
these  two  strategies  is  when  the  terminal  decision  is  made.  The  general  decision 
process  is  illustrated  schematically  in  Figure  4.3  . 
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Figure  4.3  Schematic  View  of  a  Decision  Process  (adapted  from  Ref.  12). 

1 .  Single-Stage  Strategy 

A  test  or  test  battery  is  employed  to  all  candidates.  Based  on  a  predetermined 
cutting  score,  the  terminal  decision  is  made.  Individuals  are  either  selected  or  rejected 
and  no  further  decisions  are  made.  This  strategy  corresponds  to  selecting  candidates 
randomly  or  by  a  medical  examination  only  in  pilot  selection. 

2.  Multi-Stage  Strategy 

Terminal  decisions  are  made  only  from  the  view  point  of  a  decision  maker  in 
multi-stage  strategy.  The  cutting  score  on  a  test  battery  may  be  used  to  make 
investigatory  decisions.  This  decision  calls  for  additional  information,  dictating  what 
test  or  procedure  will  be  used  to  gather  that  information.  This  then  leads  to  a  further 
decision.  The  investigatory  decisions  may  continue  through  a  number  of  stages  of 
subsequent  testing  until  terminal  decisions  are  made  regarding  all  applicants.  This  type 
of  decision  process  is  used  to  select  pilot  candidates  using  one  or  more  experimental 
testing  method  until  a  decision  maker  is  satisfying. 
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E.  THE  EXPECTED  UTILITIES  (EU) 

As  discussed  at  the  beginning  of  this  Chapter,  utility  provides  a  consistent  way  to 
compare  each  different  selection  method.  The  comparison  would  be  accomplished  by 
expected  utility,  i.e.,  maximizes  the  expected  utility  across  the  possmle  outcomes.  If 
the  four  possible  outcomes  such  that  valid  positive,  false  positive,  valid  negative  and 
false  negative  have  utility  L'Vp  ,  Ufp  .  U'vn  and  L'fn  respectively,  then  the  expected 
utility  can  be  described  generally: 

EU  =  L\.p  x  P( VP)  +  Lfp  x  P(FP)  +  Uvn  x  P(VN)  +  Ufn  x  P(FN)  -  UT 

where  Ur  is  the  utility  of  testing  method  employed.  Under  different  decision  strategies, 
the  expected  utility  should  be  considered  separately  so  that  we  can  figure  out  which 
decision  strategy  is  better  with  which  testing  method  such  that  expected  utility  for  all 
possible  outcomes  can  be  maximized. 

A  number  of  different  decision  strategies  can  result  from  combining  different 
testing  procedures.  In  the  ROKAF  selection  problem,  consideration  in  given  to  two 
simple  decision  strategies,  i.e.,  single-stage  decision  with  random  or  medical 
examination,  or  double-stage  decision  with  medical  and  one  experimental  test.  This  is 
based  on  the  assumption  that  ROKAF  has  only  two  alternatives. 

1.  Single-Stage  Decision  with  Random  Selection 

If  the  Korean  Air  Force  uses  a  random  selection  process,  providing  that  the 
SR.  BR,  number  of  applicants  N,  then  the  expected  utility  is  going  to  be: 

EU  =  Uvp  x  P(VP)  +  Ufp  x  P(FP)  +  Uvn  x  P(VN)  +  Ufn  *  P(FN» 

where  P(VP)  =  BR  *  SR  since  rp  =  0 
P(FP)  =  SR  -  P(VP) 

P(FN)  =  BR  -  P(VP) 

P(VN)  =  (1  -  BR)  -  P(FP)  =  (1  -  SR)  -  P(FN)  and 
Uj  _  0  since  no  testing  employed. 

Whenever  the  ROKAF  has  data  available  this  calculation  gives  the  ROKAF  a 
consistent  way  to  compare  alternatives  with  their  utilities.  For  the  present  effort,  it  is 
possible  to  simulate  ROKAF  data  and  evaluate  the  results.  Suppose  ROKAF  data  is 
as  follows: 
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•  SR  =  .6,  BR  =  .5.  N  =  500, 

•  utility  of  valid  positive,  in  dollar,  L'Vp  =  S80.000, 

•  utility  of  false  positive.  Uj-p  =  -  S80.000, 

•  utility  of  valid  negative,  Uvn  =  S50.000. 

•  utility  of  false  negative,  Ufn  =  0, 

Now  we  can  compute  the  expected  utility  of  a  random  selection  process  when 
the  ROKAF  uses  random  selection  of  KAFA  cadets; 

P(YP)  =  BR  x  SR  =  .5  x  .6  =  .3 

P(FP)  =  SR  -  P(VP)  =  .6  -  .3  =  .3 

P(FN)  =  BR  -  P(VP)  =  .5  -  .3  =  .2 

P(VN)  =  (1  -  SR)  -  P(FN)  =  (1  —  .6)  —  .2  =  .2 


then 


EL'  =  80,000  x  .3  -  80,000  x  .3  +  50,000  x  .2  +  0  x  .2 
=  S  10.000  per  candidate 


2.  Single-Stage  Decision  with  Medical  Examination 

When  a  medical  examination  is  employed  the  probability  of  valid  positive 

becomes: 


P(VP)  =  PS  x  SR 

where  PS  is  proportion  of  satisfactory  candidates  among  those  selected  by  a  test. 
Taylor  and  Russell  developed  a  table  for  this  PS  value  and  Appendix  C  provides  these 
values  as  different  BR  and  SR  which  is  similar  to  Taylor  and  Russell  tables  but  more 
appropriate  for  point-biserial  correlation,  r  ,  by  Abrahams.  Alf  and  Wolfe  [Ref.  10|.  If 
rp  is  .25,  then  the  PS  =  .582  with  SR  =  .6,  BR  =  .5  at  Table  given  in  Appendix  C. 

Assuming  the  utility  of  medical  examination  is  S50  then  the  expected  utility 
becomes: 

P(VP)  =  PS  x  SR  =  .582  x  .6  =  .3492 
P(FP)  =  SR  -  P(VP)  =  .6  -  .3492  =  .2508 
P(FN)  =  BR  -  P(VP)  =  .5  -  .3492  =  .1746 
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P(VN)  =  (1  -  SR)  -  P(FN)  -  (1  -  .6)  -..1746  -  .2254 


thus, 


EU  =  80,000  x  .3492  -  80,000  x  .2508  +  50,000  x  .2254  -  50 
=  519,092  per  candidate 

which  is  more  than  the  previous  random  selection  with  the  amount  of  S9,092  utility 
using  only  a  medical  examination.  This  result  indicates  that  selection  via  the  medical 
examination  is  more  effective  than  random  selection.  In  other  words,  the  medical 
examination  increases  a  certain  amount  of  utility  of  possible  personnel  selection 
outcomes. 

3.  Double-Stage  Decision  with  Medical  and  an  Experimental  Test 

Since  the  terminal  decision  is  made  via  medical  and  experimental  test 

decisions,  it  is  reasonable  to  select  enough  candidates  at  the  first  decision  so  as  to  meet 

the  required  quota  at  the  second  decision.  In  the  second  decision,  the  terminal  decision 

is  made  concerning  the  provisionally  accepted  applicants  on  the  basis  of  the  first  test. 

If  we  assume  that  SR  =  .7  in  this  case  and  r  =  .8  from  the  table  with  the  same  BR 

P 

then  the  value  of  PS  equals  .709.  With  the  utility  of  the  experimental  test  equal  to  S80, 
the  calculation  of  the  expected  utility  is,  therefore,  as  follows: 

P(VP)  =  PS  x  SR  =  .709  x  .7  =  .4963 

P(FP)  -  SR  -  P(VP)  -  .7  -  .4963  =  .2037 

P(FN)  -  BR  -  P(VP)  -  .5  -  .4963  *  .0037 

P(VN)  -  (1  -  SR)  -  P(FN)  -  (1  -  .7)  -  .0037  =  .2963 


then 


EU  -  80,000  x  .4963  -  80,000  x  .2037  +  50,000  x  .2963  -  50  -  80 
-  S38.093  per  candidate 

which  is  more  than  three  times  the  random  selection  utility  and  more  than  519,000 
utility  per  candidate  via  medical  test  only.  This  result  shows  that  if  the  ROKAF 
changes  the  present  pilot  selection  method  and  uses  an  experimental  test  method  with 
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medical  examination,  the  increased  expected  utility  o.f  that  change  is  about  SI 9,000  per 
candidate  even  if  using  small  scale  simulated  data. 

In  other  words,  the  potential  advantages  in  changing  the  present  ROKAF 
pilot  selection  system  are  obviously  considerable. 

F.  CAPER  MODEL  EVALUATION 

To  this  point  decision  strategies  have  been  discussed  in  terms  of  their  utilities  and 
advantages.  Managers  of  personnel  systems,  however,  justifiablely  demand  an  estimate 
of  the  payoff,  in  dollars,  which  can  be  expected  to  result  from  the  implementation  of  a 
proposed  selection  program.  It  might  appear  that,  for  a  given  test  validity  and  base 
rate,  a  personnel  manager  always  should  lower  the  selection  ratio  (i.e,  become  more 
selective)  this  strategy  may  or  may  not  be  cost  effective  since  lowering  selection  ratio 
causes  expanding  recruit  and  selection  efforts  to  meet  the  required  quota. 

The  purpose  of  this  section  is  again  to  evaluate  whether  the  changes  are  still 
feasible  in  terms  of  cost-effectiveness  by  using  the  Cost  of  Attaining  Personnel 
Requirements  (CAPER)  model.  Sands  [Ref.  1 1]  defined  the  CAPER  model  as  follows: 

•  "The  CAPER  model  is  designed  to  evaluate  the  cost  consequences  of  alternative 
recruiting-selection  strategies.  Specifically,  the  CAPER  model  determines  an 
optimal  recruiting-selection  strategy  for  minimizing  the  estimated  total  cost  of 
recruiting,  selecting,  inducting,  and  training  a  sufficient  number  of  persons  to 
meet  a  specified  quota  of  satisfactory  personnel.  In  addition,  the  CAPER 
model  considers  the  cost  of  an  erroneous  acceptance  (selecting  a  person  for  a 
training  program  who  subsequently  fails  to  graduate)  and  the  cost  of  erroneous 
rejection  (rejecting  a  person  who  would  have  succeeded  if  given  the 
opportunity)." 

Suppose  that  the  ROKAF  General  Staff  provides  specific  information  for  pilot 
selection  policy  such  that  the  quota,  the  base  rate  and  cost  data  for  selection, 
recruiting,  induction,  training,  erroneous  acceptance  and  rejections,  etc.  Example  of 
input  data  for  the  ROKAF  is  provided  in  Table  8  [Ref.  13]  and  shows  that  500  of  1,000 
persons  graduated,  indicating  a  .500  base  rate.  Suppose  the  data  was  given  as  follows: 

•  the  cost  of  recruiting  an  individual  is  S4.000, 

•  the  cost  of  the  present  selection  procedure  (medical  examination)  is  SI 50 
including  the  physicians'  salaries  and  laboratory  fees, 

•  the  cost  of  administering  and  scoring  the  experimental  test  is  S100, 

•  the  induction  cost  per  individual  is  estimated  as  SI 50, 

•  the  cost  for  training  is  S3, 500, 

•  the  cost  of  erroneous  acceptance  is  set  at  SI, 000,  and 

•  the  cost  of  erroneous  rejection  is  estimated  as  S800. 
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TABLE  8 

EXAMPLE  CAPER  MODEL  INPUT  DATA  FOR  ROKAF 


Graduates 


Failures 


Qualified  for  Acceptance 


Score  1  Frequency  |  Number  Proportion  Frequency  |  Number  Proportion 


Given  the  simulated  data,  the  ROKAF  can  plug  into  the  CAPER  model  the 
equations  and  compute  costs  the  present  ROKAF  selection  process  and  the  new 
selection  process.  The  set  of  equations  is  used  once  for  each  possible  cutting  score  on 
the  new  selection  process.  A  cutting  score  of  eight  on  the  test  will  be  used  for 
illustration. 

Equation  (1)  gives  the  formula  for  estimating  the  number  of  applicants  who  must 
be  recruited  in  order  to  meet  the  quota  : 

NR  =  Q  [  (BRXPG;)  ]  BR  >  0,  PG  >  0  (1) 

where  NR  is  the  number  recruited, 

Q  is  the  quota  of  satisfactory  personnel, 

BR  is  the  base  rate,  and 

PGj  is  the  proportion  of  graduates  who  would  qualify  for  acceptance  at  the  <th 
cutting  score  on  the  test. 

Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 
( 1 )  gives: 


NRp  =  50/  [  (0.50X1.00)  ]  =  100 
Similarly,  for  the  experimental  selection  procedure  (  i-  8): 

NRe  =  50/  [  (0.50X0.92)  ]  =  109 

Equation  (2)  gives  the  formula  for  estimating  the  number  of  erroneous 
acceptances  : 


NEA  =  NR(1  -  BR)PF; 


(2) 


where  NEA  is  the  number  of  erroneous  acceptances, 

PFj  is  the  proportion  of  failures  who  would  qualify  for  acceptance  at  the  /th 
cutting  score  on  the  test,  and  the  remaining  symbols  are  defined  above. 

Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 
(2)  gives: 


43 


NEAp  =  (100X1  “  0.50)(1.00)  =  50 
Similarly,  for  the  experimental  selection  procedure  ( i=  8): 

\EAe  »  (109X1  -  0.50X0.532)  =  29 


Equation  (3)  gives  the  formula  for  estimating  the  number  of  erroneous  rejections: 


r 


NER 


(NRXBRXl  ~  PG.) 


(3) 


where  NER  is  the  number  of  erroneous  rejections,  and  the  remaining  symbols  are 
defined  above. 

Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 

(3)  gives: 


NERp  =  (100X0.50)(1  -  1.00)  -  0 
Similarly,  for  the  experimental  selection  procedure  (  i  =  8): 

NERe  *  (109X0.50X1  “  0.92)  *  4 

Equation  (4)  gives  the  formula  for  estimating  the  number  of  persons  who  will 
accepted  : 


NA  «  Q  +  NEA 


(4) 


where  NA  is  the  number  accepted  and  the  others  are  the  same  as  above. 

Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 
(4)  gives: 


NA_  -  50  +  50  -  100 
P 

Similarly,  for  the  experimental  selection  procedure  ( /*  8): 
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NAe  =  50  +  29  =  79 

The  total  cost  of  the  present  selection  procedure  and  the  experimental  selection 
procedure  should  be  calculated  by  separate  equation. 

Equation  (5a)  gives  the  formula  for  estimating  the  total  cost  of  employing  the 
present  selection  procedure  to  meet  the  quota  of  satisfactory  personnel  : 

TCp  =  [  (NR)(CR)  1  +  [  (NRXCP)  1  +  [  (NA)(C1;  j 

+  [  (NA)(CT)  ]  +  [  (NEAXCEA)  +  (NER)(CER)  ]  (5a) 

where  TCp  is  the  total  cost  of  using  the  present  selection  procedure  to  meet  the  quota. 
CR  is  the  cost  of  recruiting  a  person. 

CP  is  the  cost  of  administering  the  present  procedure. 

Cl  is  the  cost  of  inducting  a  person, 

CT  is  the  cost  of  training  a  person, 

CEA  is  the  cost  of  an  erroneous  acceptance, 

CER  is  the  cost  of  an  erroneous  rejection,  and  the  others  are  the  same  as  above. 
Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 
(5a)  gives: 


TCp  =  [  (100)(S4000)  ]  +  [  (100XS1500)  ]  +  [  (100XS100)  ] 
+  ( (100XS35000)  j  +  [  (50X58000)  +  (0XS150O)  ] 

=  S3,619,120 


Equation  (5b)  gives  the  formula  for  estimating  the  total  cost  of  employing  the 
experimental  selection  procedure  to  meet  the  quota  : 

TCe  *  [  (NRXCR)  ]  +  [  (NR)(CP)  ]  +  [  (NR)(CE)  ]  +  [  (NA)(C1)  ] 

+  ( (NAXCT)  ]  +  [  (NEAXCEA)  +  (NERXCER)  ]  (5b) 

where  TCg  is  the  total  cost  of  using  the  experimental  selection  procedure  to  meet  the 
quota,  CE  is  the  cost  of  administering  the  experimental  procedure,  and  the  others  are 
the  same  as  above. 

Using  Equation  (5b)  for  the  experimental  selection  procedure  ( i-  8)  gives: 
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TCe  =  [  (I09KS4000)  1  +  [  (109KS1500)  j  +  [  (109KS80)  ]  +  [  (79XS100)  ] 
+  (  (79XS35000)  1  +  [  (29XS8000)  +  (4)(S1500)  ] 

=  S75 1,920 


A  useful  program  for  various  data  values  is  provided  in  Appendix  B  and  creates  a 
sample  CAPER  model  output  data  for  ROKAF  in  Table  9  for  the  different  values. 
This  output  contains  five  types  of  information  : 

1 )  number  recruited, 

2)  number  of  erroneous  acceptance, 

3)  number  of  erroneous  rejections, 

4)  number  accepted, 

5)  total  cost. 

These  five  estimates  may  suffice  for  many  personnel  program  managers.  And  also 
contains  cost  for  each  of  the  following: 

1 )  recruiting 

2)  selection 

3)  induction 

4)  training,  and 

5)  erroneous  decisions. 

Table  9  shows  that  as  the  cutting  score  increases  the  subsequent  results  are  as 
follows  : 

1 )  a  greater  number  of  persons  must  be  recruited, 

2)  a  smaller  number  of  persons  is  accepted, 

3 )  the  number  of  erroneous  acceptances  decreases,  and 

4)  there  is  an  increase  in  erroneous  rejections. 

These  four  consequences  have  cost  complications.  As  recruiting  and  selection  costs 
increase,  induction  and  training  costs  are  decreased.  The  ccst  of  erroneous  acceptances 
and  erroneous  rejections,  decreases  at  first,  hits  the  cutting  score  that  minimizes  the 
sum  of  both  costs  (  im  11)  and  then  increases  as  the  cutting  score  is  raised  farther  and 
the  total  cost  changes  similarly.  Figure  4.4  graphically  shpws  this  changes  of  total  cost 
as  increasing  the  cutofT  scores. 

If  the  ROKAF  wants  to  achieve  50  graduates  at  a  minimum  total  cost, 
comparing  the  estimated  total  cost  of  the  present  selection  procedure  (S4.460.000)  in 
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TABLE  9 

EXAMPLE  OF  CAPER  MODEL  OUTPUT  DATA  OF  ROKAF 


TJr~ 

SCORE 

Ht  ! 
REC  1 

NO 

ACC. 

NO 

ERR. 

ACC. 

ns  r 

ERR .  ! 

i 

REJ  .  I 

COST 

1 

1 

RECRUIT  1 

SELECT  1 

INDUCT 

i 

TRAINING  1 

ERR. DEC. 

1 

TOTAL  1 

♦ 

MA 

100  1 

100 

50 

0  1 

400000 

150000.  1 

10000 

! 

3500000 . 

400000. 

1 

4460000.  1 

0 

100  1 

100 

50 

0  1 

400000 . 

15*000.  1 

10000. 

i 

5500000 . 

400000  . 

1 

4468000.  1 

1 00  1 

130 

50 

0  1 

400000 . 

15*000.  1 

10000. 

1 

3500300 . 

400000  . 

1 

4463000  1 

2 

loo  : 

99 

49 

0  i 

400000 • 

1 53  000  1 

9900  . 

i 

3465000 . 

392000  . 

1 

4424900.  ! 

5 

100  1 

95 

43 

0  1 

400000. 

15*000.  1 

9800  . 

1 

3430000 . 

384000  . 

1 

4381800.  1 

4 

100  1 

97 

47 

0  1 

400000. 

158000.  1 

9700. 

i 

3395000 . 

376000  . 

1 

4338700.  1 

5 

100  j 

94 

o  i 

430000 . 

158000.  1 

9400 

i 

5240000 . 

352000  . 

1 

4209400.  1 

102  1 

90 

40 

1  i 

408000 . 

161160.  t 

9000  . 

i 

3150C00 . 

521500. 

1 

4049660.  1 

7 

10*  I 

35 

35 

2  1 

416300  . 

166520.  1 

8500  . 

i 

297  5000 . 

283000. 

1 

3846823.  ! 

s 

109  1 

79 

29 

4  i 

436  300  . 

172220  1 

7900  . 

i 

2765GQ0 . 

238000. 

1 

3619120.  1 

9 

113  i 

73 

23 

9  1 

472300  . 

♦  186440.  I 

7300. 

1 

2555000 . 

197500. 

1 

3418240.  1 

10 

154  i 

Q  7 

17 

17  l 

536000  . 

211720.  1 

6700 

1 

2345000 . 

161500. 

1 

5260920.  1 

1  1 

163  1 

12 

32  j 

6  52300  . 

257540.  1 

6200. 

i 

2170000 

144000 

1 

5229790  1 

12 

214  | 

59 

9 

57  j 

8  56000  . 

55*120  1 

5900  . 

1 

2065000  . 

157500 . 

1 

342252C.  1 

1 ; 

501  i 

56 

131  I 

1204030 

475580.  1 

5600. 

i 

1960000 . 

199500. 

1 

3344680.  I 

14 

46  3  : 

54 

131  I 

13  520  00  . 

751540.  1 

5400  . 

i 

1*90000 . 

505500. 

1 

4782440.  1 

15 

794  j 

52 

2 

347  | 

31  76000  . 

1254520.  1 

5200  . 

i 

1320000 . 

536500. 

1 

6792220.  1 

1471  | 

50 

0 

685  I 

5834000  . 

25241*0.  | 

5000  . 

i 

1750000 

1027500 . 

1 

10990680.  1 

r  ? 

2941  | 

50 

0 

1421  j 

1  1764000  . 

4646780.  1 

5000  . 

i 

1750000 . 

2151500. 

1 

202972*0.  1 

is 

7143  j 

50 

0 

3521  j 

23572000  . 

112*5*40.  1 

5000  . 

i 

1750000 . 

52*1500 

1 

46894416.  1 
♦ 

Table  9  suggest  that  using  the  experimental  testing  procedure  with  a  cutting  score  of  8 
(S3.619.120)  would  be  cost  effective  which  is  the  same  result  as  the  utility  evaluation. 

Table  9  shows  that  the  minimum  total  cost  of  attaining  the  quota,  50  graduates, 
is  S3, 229, 740  using  a  cutting  score  1 1  on  the  test  and  the  optimal  recruiting-selection 
strategy  is  to  recruit  163  persons.  The  best  estimate  is  that  62  of  these  persons  will 
qualify  for  acceptance  and  12  of  the  selectees  can  be  expected  to  fail  the  training 
course,  leaving  the  50  graduates  required  to  meet  the  required  quota.  In  comparison 
with  the  present  selection  procedure,  the  use  of  the  experimental  selection  procedure 
will  save  an  estimated  SI, 230, 260  or  S 24,605. 20  per  graduate. 


47 


cun, MG  SCORE 


a 


X 

CO 


•0 

o 

X 

(O 


</> 

V— 

O  * 

O  2 


o 


o  U- 
0 


4  3  12  16 

CUTTING  SCORE 

Figure  4.4  Graphical  view  of  CAPER  model  output. 
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V.  DISCUSSION 


The  increasing  number  of  pilot  candidates  entering  the  ROKAF  has  resulted  in 
increasing  costs  due  to  high  attrition  rates  and  decreasing  quality  of  pilots  who  do 
complete  the  program.  Further,  training  funds  and  personnel  resources  remain  limited. 
Solving  these  problems  will  not  be  an  easy  task.  It  appears  that  there  is  not  a  single 
simple  solution.  However,  it  is  imperative  that  some  restructuring  of  the  selection 
process  be  initiated  as  soon  as  possible  as  the  potential  negative  impact  these  problems 
will  have  continues  to  increase.  It  is  hoped  that  the  current  "Operations  with 
Economic  Efficiency  (OEE)  Program",  which  is  attempting  to  maximize  the  cost 
effectiveness  of  all  ROKAF  operations  will  be  able  to  adequately  address  the  pilot 
selection  problems.  This  thesis  represents  an  attempt  to  suggest  an  effective  method 
for  improving  the  selection  process. 

As  was  discussed  in  Chapter  I,  other  nations  -  France,  Italy  and  England  - 
initiated  the  selection  testing  efforts  as  early  as  World  War  I  and  the  L'SN  (LSAF) 
reduced  the  attrition  rate  from  60%  to  30%  (25%)  using  psychological  testing  in  the 
early  1970s.  The  Pensacola  1000  aviator  study  found  that  psychological  and 
psychomotor  measures  had  more  validity  for  the  prediction  of  success  in  flight  training 
than  did  physiological  measures.  The  psychomotor  testing  scores  are  significantly 
related  to  flight  training  pass  fail  criteria.  The  ROKAF  pilot  selection  system, 
however,  only  depends  on  actual  flight  performance. 

As  we  evaluated  its  potential  benefits  to  the  ROKAF  in  Chapter  IV,  the  expected 
utilities  and  CAPER  model,  could  have  a  significant  impact  on  pilot  selection,  pilot 
training  and  aircraft  accidents.  In  terms  of  the  analytic  CAPER  model,  where  output 
data  are  fixed  by  the  values  of  the  input  data,  the  utility  of  the  CAPER  output  can  be 
no  better  than  the  accuracy  of  the  input  data.  If  the  cost  estimates  are  inaccurate,  cost 
forecasts  and  the  optimal  recruiting-selection  strategy  will  be  distorted.  This  may  not 
be  a  serious  problem  since  the  approximations  are  usually  adequate  for  the  kinds  of 
decisions  that  estimates  are  used  to  make,  and  there  is  no  need  for  utility  estimate 
down  to  the  last  dollar. 

It  has  been  shown  that  a  greater  number  of  pilot  candidates  attrite  during  actual 
flight  training  than  during  the  pre-training  medical  screening  process.  These 
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candidates,  though  often  having  the  option  to  continue  their  military  careers  in  other 
aviation  related  fields,  frequently  feel  that  they  are  inferior  officers  because  of  the 
failure  during  flight  training.  These  feelings  of  low  self  esteem,  morale  and  motivation 
often  result  in  poor  performance  and  the  involuntary  termination  of  the  officer's  career. 
It  is  suggested  that  if  a  viable  psychological  test  battery  was  administered  in  addition 
to  medical  screening  and  a  candidate  failed  that  test  then  (1)  there  would  be  a 
significant  savings  in  actual  flight  training  time  for  that  candidate  and  (2)  his  being 
identified  as  not  aeronautically  adaptable"  might  not  have  the  psychological  impact  as 
having  failed  because  of  poor  performance  during  actual  training  flights. 

A.  RECOMMENDATIONS 

This  study  which  is  one  of  the  OEE  programs  planned  to  reduce  the  cost  and 
improve  the  efficiency  of  all  ROKAF  operations  was  essentially  designed  to  emphasize 
the  importance  and  necessity  of  mental  testing  methods  on  pilot  selection  for 
successfully  screening  out  potential  failures  and  suggests  useful  pilot  screening  methods 
to  ROKAF.  Also  provided  are  evaluation  techniques  for  decision  makers  which  are 
the  decision  strategies  and  CAPER  model. 

It  is  clear  from  the  overall  study  results  that  the  use  of  psychological  tests  or  test 
batteries  of  other  nations  can  improve  the  prediction  outcomes  of  the  ROKAF  pilot 
candidate  selection  method.  Many  other  national  Air  Forces  have  used  psychomotor 
screening  to  select  their  pilot  candidates  and  continue  efforts  to  develop  this 
methodology.  Some  of  the  psychomotor  scores  are  significantly  different  between 
candidates  who  graduate  from  UPT  and  those  who  are  eliminated.  The  ideal  selection 
system,  however,  would  measure  and  consider  all  relevant  characteristics  of  a 
candidate.  The  UPT  selection  system  currently  used  by  the  ROKAF  does  not  consider 
all  relevant  information  about  candidates  and  many  candidates  fail  training  because  of 
deficiencies  in  flying  aptitude.  The  intense  situation  of  ROKAF,  moreover,  requires 
selecting  high  quality  student  pilots  and  the  candidates  with  the  lowest  probability  of 
success  would  be  screened  out  before  training.  The  higher  the  cutoff  point  is  set,  the 
better  the  pilot  quality,  cost  effectiveness  not  with  standing.  The  "false  rejection", 
however,  is  a  problem  with  all  screening  systems,  including  flight  screening,  and  must 
be  evaluated  in  terms  of  the  percentage  screened  and  the  quality  of  the  graduating 
students. 

As  suggested  in  Chapter  IV,  a  change  using  experimental  testing  on  ROKAF 
pilot  selection  method  is  feasible.  It  suggests  a  savings  of  S24.605.20  per  graduate  in 
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our  example.  Multitask  performance  is  useful  .for  the  primary  flight  training 
performance  prediction.  The  experiment  takes  only  20  minutes  and  is  easy  to 
administer.  The  psychomotor  screening  system  used  by  the  USAF,  which  includes 
Two-Hand  Coordination  and  Complex  Coordination,  has  been  shown  to  be  useful  in 
screening  not  only  candidates  for  L  PT  but  also  for  identifying  those  who  demonstrate 
a  potential  for  successfully  completing  advanced  fighter  training.  Any  of  three 
Integrated  Pilot  Candidate  Selection  Models  could  be  used  to  select  LPT  candidates  in 
the  same  manner  as  the  psychomotor  screening  system.  The  whole  person  concept  is 
imbedded  in  the  multidimensional  IPCSM  approach  to  pilot  selection. 

The  implementation  of  psychomotor  screening  and  multitask  performance  will 
improve  the  quality  of  student  candidates  and  reduce  attritions  in  ROKAF  LPT. 
Based  on  overall  considerations,  however,  an  integrated  selection  system  including 
psychomotor  screening  is  recommended  for  ROKAF. 

One  of  the  preferable  pre-selection  systems  that  suggests  for  ROKAF  pilot 
selection  system  of  AFA  cadets  in  this  study  would  be: 

1)  The  General  Staff  defines  the  number  of  pilots  needed,  and  a  circular  is  issued 
identifying  the  number  of  candidates  required. 

2)  The  selection  process  includes: 

a)  Preliminary  Examination  -  Medical 

b)  EEG  recordings  (Electroencephalogram) 

c)  Integrated  Psychological  Test 

d)  Individual  interview. 

3)  Cadets  must  wait  until  after  their  graduation  from  the  KAFA  and  commission 
as  officers  before  starting  flight  training. 

Other  useful  methods  which  might  also  be  incorporated  in  a  pre-selection  screening 
process  would  be: 

•  Perceptual/Cognitive  Paper-and- Pencil  Tests, 

•  Neurological  Test, 

•  Residual  Attention  Test. 

The  Residual  Attention  Test  could  be  especially  valuable  in  predicting  how  well  an 
individual  will  handle  emergency  situations.  This  could  potentially  have  an  impact  on 
the  aircraft  accident  rate  in  accidents  due  to  human  ersor,  incorrect  procedures  or  just 
poor  handling  of  the  aircraft  during  emergencies.  It  is  obvious  that  with  the  many 
types  of  individual  psychological  and  psychomotor  tests  available  that  further  research 
must  now  address  which  individual  tests  and  combinations  of  tests  and  batteries  of 


51 


must  now  address  which  individual  tests  and  combinations  of  tests  and  batteries  of 
tests  should  be  incorporated  into  the  ROKAF  selection  program.  It  is  strongly 
recommended  that  a  ROKAF  pilot  selection  program  research  and  development  (R  & 
D)  team  be  organized  to  address  this  and  other  selection  issues.  The  team  could  also 
benefit  greatly  from  a  close  liaison  with  L\S.  Navy  and  Air  Force  R  &  D  centers 
currently  engaged  in  aircrew  selection  methodology. 

It  is  hoped  that  this  study  will  be  useful  in  stimulating  the  ROKAF  in  developing 
improved  selection  methods  and  in  predicting  success  in  training  as  a  means  of 
increasing  aircrew  training  program  cost  effectiveness. 
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List  of  Abbreviation 


ACTB 

Aircrew  Classification  Test  Battery 

A  FA 

Air  Force  Academy 

AFB 

Air  Force  Base 

AFHRL 

Air  Force  Human  Resources  Laboratory  (AFSC) 

AFOQT 

Air  Force  Officer  Qualifying  Test 

AFROTC 

Air  Force  Reserve  Officer  Training  Corps 

AOC 

Aviator  Officer  Candidate 

AQT 

Aviation  Qualification  Test 

ASTB 

Aviator  Selection  Test  Battery- 

FAR 

Flight  Aptitude  Rating 

FR 

Fighter  Recommended 

FSP 

Flight  Screening  Program  (OTS) 

IPCSM 

Integrated  Pilot  Candidate  Selection  Model 

KAFA 

Korea  Air  Force  Academy 

OEE 

Operations  with  Economic  Efficiency 

OTS 

Officer  Training  School 

ROKAF 

Republic  of  Korea  Air  Force 

R&D 

Research  and  Development 

LPT 

Undergraduate  Pilot  Training 

LSAF 

United  States  Air  Force 

LSN 

United  States  Navy 
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APPENDIX  A 

ATTRITION  CATEGORIES 


A  student  pilot  can  be  dropped  from  a  training  program  for  several  reasons.  It  is 
agreed  that  there  is  no  one  simple  causative  factor  nor  a  quick  and  easy  solution. 
Some  of  the  primary  reasons  for  student  attrition  are  : 

/.  Voluntary  Withdrawal  or  Drop  On  Request  (DOR) 

Over  50  research  reports  and  findings  associate  the  main  reason  of  DOR 
with  anxiety,  tension  or  fear.  Most  of  the  specific  reasons  for  the  DOR  are  expressed 
an  anxiety  or  tension  with  regard  to  flying,  fear  of  flying,  or  lack  of  confidence  in 
handling  aircraft.  Also  mentioned  are  a  dislike  or  distaste  of  flying  and  a  shift  to  a 
civilian  career  interest.  Some  of  the  researches  indicate  a  connection  of  fear  of  aircraft 
accidents  or  crashing  with  failure  in  training. 

2.  Not  Phisically  Qualified 

Most  research  suggests  that  "a  person  with  anxiety  may  see  the  flight 
surgeon  when  he  has  a  symptom  which  he  knows  might  result  in  his  being  dropped 
from  the  program;  whereas,  a  nonanxious,  highly  motivated  person  might  never  go  to 
the  flight  surgeon." 

3.  Flight  Failure  (FF) 

Figure  1.1  mentions  the  types  of  attrition  by  stage  of  training.  The  data 
provide  dramatic  evidence  of  the  relationship  of  DOR  and  FF  in  primary  training  and 
the  transition  phase. 

The  research  literature  generally  supports  the  view  that  anxiety  is  both  a 
cause  and  a  symptom  of  inadequate  or  inferior  performance  in  training.  It  can  be 
broken  down  into  two  subcategories: 

1)  A  student  group  prone  to  anxiety  or  with  anxiety  predisposition. 

2)  A  student  group  going  through  a  transient  anxiety  producing  situation. 

4.  Not  Officer  Material 

Is  a  statistic  more  oriented  to  the  time  during  which  attrition  occurs  rather 
than  a  specific  type  of  attrition. 

5.  Academic  Failure 

Unsatisfactory  performance  in  academic  work. 
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6.  Transfer  to  Another  Training  Program  ■ 

This  is  a  small  percentage  of  the  total  attrition  and  means  a  request  to 
change  one's  specialty. 

7.  Practical  Work  Failure 

Unsatisfactory  performance  in  a  ground  simulator  or  training  device. 
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APPENDIX  B 

CAPER  MODEL  PROGRAM 


************************************************************************ 
*  * 

*  CAPER  MODEL  * 

*  ★ 

*  This  program  provides  a  CAPER  MODEL  OUTPUT  as  various  input  data.  * 

*  The  program  reads  data  from  data  file,  CAPER  DATA,  and  creates  a  * 

*  formated  output,  CAPEROU  LISTING  on  your  disk  automatically  using  * 

*  exec  file.  The  data  used  are  from  the  Table  8  and  there  is  no  * 

*  no  meaning  of  ROKAF  data  but  only  for  showing  how  the  program  works.  * 


The  key  variables  used  are 


number  recruited 
number  accepted 

number  of  erroneous  acceptances 
number  of  erroneous  rejections 
the  quota  of  satisfactory  personnel 
base  rate 

proportion  of  graduates  who  would  qualify  for 

acceptance  at  the  ith  cutting  score  on  test 

proportion  of  failures  who  would  qualify  for 

acceptance  at  the  ith  cutting  score  on  test 

cost  of  recruiting  a  person 

cost  of  adninistering  the  present  procedure 

cost  of  adninistering  the  experimental  procedure 

cost  of  inducting  a  person 

cost  of  training  a  person 

cost  of  an  erroneous  acceptance 

cost  of  an  erroneous  rejection 

total  cost  of  using  the  present  selection 

?rocedure  to  meet  the  quota 
otal  cost  of  using  the  experimental  selection 
procedure  to  meet  the  quota 


quota 


*  This  program  could  be  used  effectively  for  other  CAPER  MODEL  * 

*  changing  only  the  data  and  also  useful  for  interactive  program  with  * 

*  some  developments.  * 

************************************************************************ 


PARAMETER  (N-20 
REAL  PG(N),PF(N) 

DATA  NO/ 50/ , BR/ . 5/ . CR/4000 . / , 
*  CEA/80d0/ ,CER/l5u0/ 

***  READ  DATA  FROM  DATA  FILE  *** 

DO  10  1*1  .N 

READ  (1,5)  PG(I) ,PF(I) 

10  CONTINUE 

***  OUTPUT  FORMAT  *** 

WRITE(3,15) 

WRITE(3 ,65) 

65  FORMAT (IX, ' (  CUT  |  HO  |  NO 
WRITE(3,75) 

75  FORMAT (IX, 1 ( SCORE |  REC.  |  ACC. 

WRITE(3 ,*)  1 |  |  \ 

WRITE (3, 25) 

WRITE (3,*)  'I  |  I 

*ECT  |  INDUCT  |  TRAINING 


CP/1500 . / , CI/100 . / , CE/80 . / , CT/35000 . / 


ERR.  |  ERR. 


| 1 ,72X, ' | ' ) 
t ' , 34X , 1  COST ' , 34X, 1 


I  ACC.  |  REJ.  I  RECRUIT 
ERR. DEC.  |  TOTAL  I' 
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WRITE(3 , 15) 

WRITE (3,*) 

***  CALCULATE  TOTAL  COST  *** 

DO  20  J=1 ,N 

NR  =  NINT(NQ/(BR  *  PG(J))) 

NEA  =  NINT(NR  *  (1  -  BR)  *  PF(J)) 

NER  =  NINT(NR  *  BR  *  (1  -  PG(J)) 

NA  =  NO  +  NEA 

TR  =  NR  *  CR 

SP  =  NR  *  CP 

SE  =  NR  *  CP  +  NR  *  CE 

TI  =  NA  *  Cl 

TT  =  NA  *  CT 

TE  =  NEA  *  CEA  +  NER  *  CER 
TCS  =  TR  +  TI  ♦  TT  +  TE 
TCP  =  TCS  +  SP 
TCE  -  TCS  +  SE 
IF  (J.EQ.l)  THEN 

WRITE(3,35)  NR , NA , NEA , NER , TR , SP , TI , TT , TE , TCP 
WRITE (3 , 55 ) 

ELSE 

WRITE (3, 45)  (3-2) , NR ,NA ,NEA ,NER, TR, SE ,TI ,TT, TE , TCE 
END  IF 

20  CONTINUE 

WRITE (3, 15) 

5  FORMAT  (F5.3,5X,F5.3) 

15  FORMAT  (IX,  '+',106( 1  ‘  ) 

25  FORMAT  ('  +  '  T37 , 72 ( 1  -  1 ) , 1  I ' ) 

35  FORMAT  (IX,  '  NA  |  ’,4(14,'  |  ^^(FIO.O,1  |  • ) ) 

45  FORMAT (IX, 1  ',12,'  |  ',4(14,'  '),6(F10.0,‘  |')) 

55  FORMAT ( IX , '  ',T8,'|  ',4(4X,*  |  '),6(10X,'  I')) 

STOP 

END 


NR , NA , NEA , NER , TR , SP , TI , TT , TE , TCP 


***  DATA  FILE  *** 
PG  PF 


1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.996 

.984 

.962 

.920 

.848 

.744 

.612 

.468 

.332 

.216 

.126 

.068 

.034 

.014 


1.000 

1.000 

1.000 

.986 

.966 

.932 

.874 

.784 

.668 

.532 

.388 

.256 

.152 

.080 

.038 

.016 

.004 

.000 

.000 

.000 
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APPENDIX  C 

TAYLOR  AND  RUSSELL  TABLES 

X.  M.  Abakams,  E.  F.  Au,  amb  J.  H.  Worn 
TABU  l 


PwtotnoM  Vn  Wiu.  Bt  SAtatAcniy  a»n  TM  Ssucas,  m  Cnw  vaibx  of 
sb  Pioforrrox  or  Piubt  Ehflotwii  Comroun  Satotactoh,  as 
StucRoa  Kasd,  am  Posr*Bbbiaa  f 


b 

b 

.090 

.100 

200 

J00 

.400 

J00 

.000 

.700  |  J00 

.900 

.990 

Proporoa*  erf  nployaa  aammdtnd  miAaery  -  .OS 


.00 

.090 

.090 

.090 

.090 

.090 

.090 

.090 

.090 

.090 

.090 

.09 

.076 

.072 

.066 

.063 

.061 

.099 

.097 

.099 

.094 

.092 

.10 

.112 

.099 

.086 

.07$ 

.on 

.067 

.063 

.060 

.096 

.093 

.13 

.137 

.131 

.106 

.092 

.002 

.073 

.068 

.063 

.098 

.094 

.20 

213 

.169 

.128 

.107 

.092 

.002 

.073 

.066 

.060 

.099 

23 

.27$ 

.210 

.131 

.121 

.007 

.076 

.068 

.061 

.099 

.W 

J92 

.233 

.173 

.133 

.092 

.079 

.070 

.062 

.093 

J3 

.433 

.302 

.194 

.144 

.099 

.001 

.070 

.062 

.096 

.40 

JI9 

.34$ 

■212 

.193 

.097 

.062 

.071 

.062 

.096 

.49 

.607 

.393 

.227 

.199 

.122 

.099 

.003 

.071 

.062 

.096 

-90 

.694 

.431 

.23$ 

.163 

.124 

.100 

.063 

.071 

.062 

.096 

99 

.776 

462 

.249 

.169 

.124 

.100 

.083 

.071 

.062 

.096 

.60 

.$49 

.484 

.248 

.166 

.129 

.100 

.083 

.071 

.062 

.096 

.69 

.909 

.499 

-230 

.167 

.129 

.100 

.083 

.071 

.062 

.096 

.70 

.994 

499 

-230 

.167 

.129 

.100 

.063 

07t 

.062 

.096 

.79 

.982 

300 

.290 

.167 

.123 

.100 

.083 

.071 

.062 

.096 

.30 

.996 

.900 

.230 

.167 

.129 

.100 

.083 

.071 

.062 

.096 

AS 

.996 

.900 

290 

.167 

.129 

.100 

.083 

.071 

.062 

.096 

.90 

.996 

.900 

-290 

.167 

.129 

.100 

.083 

.071 

.062 

.096 

.93 

1.000 

.900 

-290  I 

.167 

.129 

100 

.083 

.071 

.062 

.096 

1.00 

1.000 

.900 

.290 

.167 

.129 

.100 

.083 

.071 

.062 

.096 

Proporooa  a I  Bpln;  —  w»d— d  lirfinwy  «  .10 


.00 

■S3 

.100 

.100 

.100 

.100 

.100 

.100 

.100 

.100 

.09 

.129 

.122 

.111 

.112 

.110 

.107 

.105 

.103 

.102 

.10 

■rri 

.162 

.147 

.137 

■120 

.124 

.118 

.114 

.109 

.109 

403 

.13 

J26 

200 

.173 

.137 

1  .149 

.139 

.127 

.120 

40 

.107 

.106 

JO 

JS4 

241 

J02 

an 

.140 

.148 

.129 

414 

408 

.106 

JS 

J46 

J90 

232 

.191 

.197 

.130 

.119 

409 

.106 

JO 

.422 

J43 

J64 

J19 

■O 

.148 

It! 

.133 

.121 

.110 

.109 

-33 

JB2 

J9* 

J97 

240 

,m 

.179 

In] 

.134 

422 

.no 

.105 

.40 

-986 

.460 

JJ0 

260 

J14 

.182 

.139 

.123 

.in 

.103 

.49 

.676 

923 

-364 

.278 

229 

.118 

.181 

.140 

.124 

.in 

.105 

JO 

.761 

J92 

J97 

298 

234 

.191 

.148 

.129 

.m 

.109 

JS 

J47 

.661 

■427 

-304 

240 

.198 

| 

.142 

.129 

.in 

.109 

.60 

J17 

.731 

494 

J20 

249 

.191 

.143 

429 

au 

409 

.69 

.668 

J00 

.474 

J27 

248 

.199 

.14) 

.123 

.lit 

.109 

.70 

.961 

.864 

.490 

JJ1 

249 

200 

'  443 

429 

.in 

.109 

.79 

.966 

.920 

.497 

-333 

290 

200 

.187 

.143 

423 

an 

.109 

.80 

1.000 

.964 

JG0 

J33 

230 

200 

.187 

.143 

429 

an 

.109 

JS 

1.000 

.990 

J00 

-333 

290 

200 

.187 

.14) 

429 

411 

409 

JO 

1.000 

.999 

J0Q 

J33 

290 

200 

.187 

.14) 

.129 

411 

409 

JS 

1.000 

.999 

J00 

-333 

290 

200 

.187 

.14) 

.129 

411 

409 

1.00 

1.000 

1.000 

900 

J33 

280 

200 

.187 

.143 

429 

411 

409 
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Tayloi-Rbssbia  Tabus  fob  Dichotomogs  -Cbrxbxon  Vakiabus 


TABLE  1— (Cento.**) 


Selection  redo 


joo  j 

J00 

.400 

soo  | 

.600  | 

.700 

.800 

.900 

Proportion  of  employee*  considered  sedsfectory  »  .20 


.00 

|  .200 

.200 

.200 

1  .200  1 

.200 

.200 

.200 

.200  i 

'  1 

.200  • 

.200  j 

.200 

05 

.244 

.237 

.229 

■  .224 

.220 

i  .216 

.213 

.210 

207 

.204 

.202 

.10 

|  .293 

•277 

.260 

.248 

.239 

|  .232 

.225 

.219 

.213 

.207  i 

204 

.15 

.347 

i  .320 

.292 

.274  ! 

.260 

.248 

.238 

.228 

.219 

.210 

205 

.20 

.405 

.368  1 

.326 

.300 

.280 

.264 

.249 

.236 

.224 

.213 

.207 

.25 

.468 

.418  ] 

.363 

1  .328  I 

.301 

.280 

.261 

.244 

.229 

.215 

.208 

.30 

.535 

.472  ! 

.401 

356 

.322 

.295 

.272 

.252 

234 

.217 

.208 

.35 

.605  ; 

.529  , 

.441 

.385  ! 

344 

.311 

.283 

.259 

.238 

218 

.209 

.40  | 

.676 

.589  j 

484 

.416 

.366 

.326 

.293 

.265 

.241 

.220  1 

210 

.45  ' 

747 

.652  . 

.529 

.448 

.387 

.340 

.302 

.270 

.244 

220 

.210 

.50  , 

.815 

.717 

.577 

480 

.409 

354 

.310 

.275 

246 

.221 

.210 

.55 

.876 

782  , 

.627  . 

514 

.430 

.367  | 

.318 

279 

248 

.222 

.210 

.60 

.927 

346 

.680  ! 

.547 

450 

.378  1 

324  i 

.282 

.249 

.222 

.210 

.65  i 

.964 

.904 

.736  1 

.581  ' 

.468 

.387  | 

328 

.284 

249 

.222 

210 

70  ! 

.986 

.952 

.794 

.612  ! 

.482  1 

.394  i 

.331  I 

.285 

.250 

222 

|  .210 

.75  1 

996 

983 

852 

639  1 

.492 

.398 

.333  i 

286 

[  .250 

.222 

|  210 

80  i 

1.000 

997 

909 

.657 

498 

.400  1 

333  1 

.286 

250 

.222 

210 

45  I 

1.000 

1.000 

.958 

.665 

500 

.400 

33 3 

286 

250 

222 

210 

JO  . 

1.000 

1.000 

990 

667 

500 

400 

333 

.286 

250 

.222 

.210 

95 

1.000 

1.000 

990 

667 

500 

.400 

333 

.286 

250 

.222 

210 

100 

1.000 

1.000 

1000 

.667  1 

.500 

.400 

333 

286 

250 

.222 

.211 

Proportion  of  employees  considered  uusfactory  *  30 


00 

.300 

.300 

300 

300 

300 

.300 

.300 

300  1 

300 

.300 

300 

05 

349 

341 

333 

327 

322 

318 

315 

311 

308 

.304 

302 

10 

401 

385 

367 

354 

345 

.337 

329 

.322 

316 

.308 

305 

.15 

456 

430 

402 

383 

368 

355 

344 

333  , 

323 

.312 

.307 

20 

.513 

478 

438 

.412 

.391 

.374 

358 

344 

330 

.316  , 

.308 

.23 

572 

528 

476 

.442 

.415 

393 

.372 

354 

336 

319  1 

.310 

30 

632 

.579 

.516 

.473 

.440 

412 

387 

364 

343 

.322 

.311 

35 

692 

632 

.557 

.506 

.465 

.431 

401 

.374  | 

349 

.324 

.312 

40 

.751 

686 

600 

.540 

.492 

.451 

415 

383  1 

.354 

.327 

314 

45 

.808 

.740 

.646 

.576 

.519 

.471 

.429 

.392 

359 

.329 

314 

SO 

.860 

.794 

.693 

.614 

.547 

.491 

.443 

.401  | 

.363 

330 

J15 

55 

.905 

.846 

.742 

.653 

.577 

.512 

456 

.408 

.367 

.331 

.315 

60 

.942 

.894 

.794 

.696 

.608 

532 

468 

.415 

370 

.332 

316 

65 

970 

.936 

846 

.741 

.639 

552 

.479 

.420 

.372 

.333 

.316 

70 

.987 

.967 

.897 

.789 

.671 

.570 

.488 

.424  j 

.374 

333 

316 

.75 

.996 

.988 

.944 

839 

.702 

584 

.495 

.427  ; 

.375 

333 

316 

80 

.999 

.997 

.979 

.891 

.728 

594 

498 

.428 

375 

333 

316 

85 

1.000 

1.000 

.997 

.941 

.745 

.599 

500 

428 

.375 

333 

316 

.90 

1.000 

1.000 

1.000 

.982 

.750 

600 

500 

429 

.375 

333 

3 16 

95 

1.000 

1.000 

1.000 

.982 

750 

600 

500 

429 

375 

333 

316 

00 

1.000 

1.000 

1.000 

1.000 

750 

.600 

.500 

.429 

.375 

333 

i 

.316 

Ptopottioc  of  employee*  coMidered  wtieftctney  *  .40 

.00 

.400 

.400 

.400 

.400 

.400 

.400 

.400 

.400 

.400 

.400  ! 

.400 

.05 

.444 

.435 

.429 

.424 

.420 

.416 

.412 

.406 

.405  ! 

405 

.10 

488 

.470 

.458 

.448 

439 

.432 

.424  ( 

.417 

.409 

405 

.15 

HUM 

.534 

.506 

.487 

.472 

459 

.447 

.436 

.425 

.414 

408 

20 

.580 

.542 

.517 

.497 

.479 

.463 

.448  ! 

1 

433 

.418 

.410 
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TABLE  HCmM) 


Selection  ratio 


'» * 

.050 

.100 

J00 

300 

.400 

.500 

.600 

.700 

m 

.950 

Proportion  of  employees  considered 

satisfactory  "  .40 

.25 

.665 

.626 

.580 

.548 

.522 

.500 

.479 

.460 

.441 

.422 

-412 

.30 

.717 

.673 

.618 

.580 

.548 

.521 

.496 

.472 

.449 

.426 

.414 

.33 

.768 

.720 

.658 

.613 

.575 

.542 

.512 

.484 

.456 

.429 

413 

.40 

.816 

.766 

.698 

.647 

.603 

.564 

.329 

.496 

.464 

.433 

417 

45 

.860 

.811 

.740 

.682 

.632 

588 

.546 

.508 

.471 

.436 

.418 

.50 

.900 

854 

.782 

.720 

.662 

.612 

.564 

.519 

.477 

.438 

.419 

.55 

.933 

.894 

.824 

.758 

.696 

.637 

.582 

.530 

.483 

.440 

.420 

.60 

.959 

.929 

.866 

.799 

.730 

.663 

.600 

.541 

.489 

442 

.420 

.65 

.978 

.958 

.906 

842 

.768 

.691 

.617 

.551 

.493 

443 

421 

.70 

.990 

.979 

.943 

.88 5 

.807 

.719 

.634 

560 

.496 

.444 

.421 

.75 

.997 

.992 

.972 

.928 

.850 

.748 

.649 

.566 

498 

444 

.42 

80 

.999 

.998 

.991 

.966 

.894 

.774 

660 

.570 

.500 

.444 

.41. 

.85 

1.000 

1.000 

.998 

.992 

939 

.792 

.665 

.571 

.500 

444 

.421 

.90 

1.000 

1.000 

1.000 

1.000 

979 

.800 

.667 

.571 

500 

.444 

.421 

95 

1.000 

1.000 

1.000 

1.000 

.979 

800 

.667 

571 

.500 

444 

421 

100 

1.000 

1000 

1  000 

1000 

1.000 

.800 

667 

.571 

.500 

.444 

.421 

Proportion  of  employee!  considered  satisfactory  m  SO 


00 

.500 

500 

500 

500 

.500 

.500 

.500 

500 

.500 

.500  ' 

500 

05 

.552 

544 

535 

529 

524 

.520 

516 

.512 

.509 

505 

503 

.10 

.602 

.588 

.570 

558 

548 

.540 

532 

525 

.518 

510 

505 

IS 

.653 

631 

603 

.587 

573 

560 

549 

538 

526 

.515 

.5,'S 

20 

.702 

674 

640 

.617 

.598 

581 

.565 

550 

535 

519 

.5  1 

.25 

.748 

.716 

676 

.647 

623 

.602 

582 

.563 

544 

524 

515 

30 

.793 

.737 

.712 

678 

649 

623 

.599 

.576 

.553 

.529 

515 

33 

.834 

.797 

.747 

709 

.676 

646 

617 

590 

562 

.533 

.518 

40 

.872 

.835 

.783 

.741 

.703 

669 

.636 

603 

.571 

.537  ! 

520 

45 

90S 

.871 

.818 

.773 

.732 

693 

.655 

617 

580 

541 

.521 

so 

.933 

.903. 

853 

.807 

762 

718 

675 

631 

588 

545 

523 

55 

.956 

.932 

886 

.840 

793 

.745 

.696 

646 

597 

548 

524 

60 

.974 

.956 

.918 

.875 

826 

.773 

718 

.660 

604 

.551 

.525 

.65 

.986 

.975 

946 

.908 

161 

804 

.740 

675 

612 

553  1 

526 

.70 

.994 

.988 

.969 

940 

896 

836 

.764 

689 

617 

554 

526 

.73 

998 

995 

986 

968 

932 

172 

.788 

700 

622 

553 

526 

80 

1.000 

.999 

.996 

.988 

.966 

909 

110 

.709 

.624 

355 

::6 

85 

1.000 

1.000 

.999 

998 

990 

.947 

.827 

.713 

625 

556 

-.6 

90 

1.000 

1.000 

1.000 

1000 

999 

980 

833 

.714 

.625 

356 

3-0 

95 

1.000 

1.000 

1.000 

1.000 

1.000 

999 

833 

.714 

625 

556 

520 

1.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

833 

.714 

625 

.556 

526 

Proportion  of  employee*  considered  saosfacxory  *  . 

to 

00 

.800 

.600 

.600 

600 

600 

.600 

.600 

too 

600 

600 

OS 

.650 

.642 

.634 

.628 

.624 

.620 

.616 

.612 

.609 

60S 

603 

.10 

.8*7 

.684 

.667 

656 

.647 

639 

.632 

625 

.618 

.610 

604 

.13 

.742 

.725 

.700 

.684 

.671 

.659 

648 

637 

.626 

.615 

.608 

.20 

.784 

.761 

.733 

.712 

.695 

179 

.664- 

.650 

616 

620 

611 

.25 

.87.4 

.798 

.764 

.740 

.719 

.700 

.681 

.661 

645 

.625 

614 

JO 

*59 

.832 

.796 

.768 

.743 

.721 

.699 

.677 

.655 

610 

617 

JS 

J9t 

.864 

126 

.796 

768 

.742 

.691 

.664 

.636 

619 

.40 

.918 

.893 

.855 

.823 

.793 

.764 

.706 

.675 

.641 

622 

.45 

.*2 

.920 

883 

151 

119 

.788 

.721 

.685 

.646 

UHL 
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Taylor- Russell  Tables  for  Dichotomous.  Criterion  Variables 


TABLE  1 — (Ctntvuui) 


Selection  redo 


'pt> 

E3 

BS 

500 

.400 

.500 

E3 

.700 

.800 

ES 

.950 

Proportion  of  employees  considered  satisfactory  -  .60 

.50 

.961 

942 

.910 

.878 

.846 

.812 

.776 

.737 

.695 

.650 

.626 

.53 

.976 

.961 

.934 

.905 

.872 

.837 

.797 

.754 

.706 

.655 

.628 

.60 

.986 

.976 

.955 

.930 

.900 

.863 

.820 

.771 

.716 

.659 

.629 

65 

.995 

.987 

.972 

.953 

.926 

.891 

.845 

.789 

.727 

.662 

.630 

.70 

.997 

.994 

.986 

.972 

.951 

.919 

.872 

.808 

.736 

.664 

.631 

75 

.999 

.998 

994 

.987 

.973 

.948 

.900 

.826 

.743 

.666 

.631 

50 

1.000 

1.000 

.998 

.996 

.990 

.974 

.930 

.843 

.748 

.666 

.632 

55 

1.000 

1.000 

1.000 

.999 

.998 

.992 

.960 

.854 

.750 

.667 

.632 

.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.986 

.857 

.750 

.667 

.632 

.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.857 

.750 

.667 

.632 

1.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.857 

.750 

.667 

.632 

Proportion  of  employees  considered  satisfactory  •  .70 


.00 

.700 

.700 

.700 

.700 

.700 

.700 

.700 

.700  i 

.700 

.700 

.700 

.05 

.746 

.739 

.732 

.726 

.722 

.718 

.715 

.712  i 

.708  [ 

.705 

.703 

10 

.787 

.776 

.762 

.752 

.744 

.737 

.730 

.723 

.717  | 

.709 

.705 

15 

.826 

.810 

.791 

.777 

.766 

.755 

.743 

.736  1 

.725 

.714 

.708 

.20 

.860 

842 

.819 

.802 

.787 

.774 

.761 

.748  1 

.735 

.720 

.711 

25 

.890 

.871 

.846 

.826 

.809 

.793 

.777 

.761  1 

744 

.725 

.714 

30 

.916 

897 

.871 

.849 

.830 

.812 

.793 

.774  1 

.754 

.731 

.718 

35 

.938 

920 

.894 

.872 

.852 

.831 

.810 

.788  | 

.764 

.737  : 

.721 

.40 

.956 

941 

.916 

.894 

.873 

.851 

.828 

.803 

.775 

743 

724 

.45 

.970 

.958 

.936 

.915 

.894 

.871 

.846 

.818  1 

.786 

.749 

.727 

50 

.981 

.972 

.953 

.935 

.914 

.891 

.865 

.834  | 

.798 

.755  . 

.730 

.55 

.989 

.982 

968 

.952 

.934 

.912 

.885 

851  : 

.811  , 

.761  ! 

.732 

.60 

994 

.990 

980 

.968 

.952 

.932 

.905 

.870 

.823 

.766 

.734 

.65 

.997 

.995 

989 

.981 

.969 

.952 

.926 

.889 

.836  ) 

.771 

.735 

70 

999 

.998 

.995 

.990 

.982 

.970 

.948 

.909 

.849 

.774  , 

.736 

.75 

1.000 

1000 

1.000 

.996 

.992 

.984 

.968 

.931 

.861  1 

.776 

.737 

so 

1.000 

1.000 

1.000 

.999 

.998 

.994 

.986 

.953 

.870  | 

.778 

.737 

65 

1000 

1.000 

1.000 

1.000 

1.000 

.999 

.997 

.975  l 

.874 

.778 

.737 

90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.992  1 

.875  1 

.778 

.737 

95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.992  1 

.875  i 

.778  1 

.737 

1  00 

1.000 

1.000 

1.000 

1.000 

1. 000 

1.000 

1.000 

1.000  , 

.875  j 

.778 

1 

.737 

Proportion  of  employees  considered  satisfactory  ”  .80 


.00 

800 

.800 

1 

.800 

.800 

1H5S 

l 

.800 

05 

.839 

.833 

.827 

.823 

.819 

.816 

.813 

.810 

802 

.10 

.872 

.864 

.852 

.844 

.838 

.832 

.826 

.821 

.815 

.809 

.805 

IS 

.901 

476 

.865 

.856 

.848 

.840 

.832 

823 

.813 

.808 

20 

.926 

914 

.898 

.885 

.874 

.864 

.853 

.843 

.819 

.811 

.25 

946 

.934 

.917 

.891 

.880 

.867 

.855 

.841 

824 

.814 

30 

.962 

.951 

.935 

.921 

.908 

895 

.882 

.867 

.850 

.830 

.818 

35 

.974 

.965 

Kin 

.937 

.924 

.911 

496 

.879 

.860 

.836 

.821 

40 

963 

976 

.964 

.952 

.939 

.926 

892 

.871 

443 

.825 

45 

984 

.975 

.964 

.953 

.940 

.925 

.906 

.882 

.850 

429 

Kl 

.994 

.991 

.984 

.973 

.966 

.954 

.939 

.894 

.857 

.832 

.55 

.997 

.995 

.984 

.977 

.967 

.934 

.907 

.865 

.836 

.999 

.998 

995 

.991 

.985 

.978 

.966 

.949 

.872 

.65 

1.000 

VTTmK 

.992 

.987 

.978 

.963 

.934 

.878 

.840 

1.000 

1.000 

1.000 

1.000 

.994 

.988 

.977 

.948 

.884 

.841 
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TABLE  I — (Contituui) 


Selection  ratio 


.050 

.100 

J00 

J00 

.400 

J00 

.600 

.700 

.800 

.900  j 

.950 

Proportion  of  employees  considered  satisfactory  -  .SO 

.75 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.995 

.988 

.965 

.887 

842 

.80 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.996 

.977 

.888 

842 

.85 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.990 

.889 

842 

.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.998 

.889 

.842 

.95 

1.000 

1.000 

1.000 

1.000 

■  KO’Jfl 

1.000 

1.000 

1.000 

.998 

.889  j 

.842 

1.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.0G0 

.889 

.842 

Proportion  of  employees  considered  satisfactory  -  .90 


.00 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

05 

.928 

.924 

.920 

.917 

.914 

.912 

.910 

.908 

.906 

.903 

.902 

10 

.949 

.944 

.937 

932 

.928 

.924 

.920 

.916 

.912 

.907 

.904 

.15 

.966 

.960 

.952 

.946 

.940 

.935 

.930 

.924 

.918 

.911 

9^: 

20 

.978 

.972 

965 

.958 

.952 

.946 

.940 

.933 

.925 

.916 

9'.J 

.25 

.986 

982 

.975 

969 

.963 

.957 

950 

.942 

933 

.921 

.913 

.30 

.992 

989 

.983 

.978 

.972 

.966 

.959 

951 

.941 

.927 

917 

.33 

.996 

.993 

.989 

.985 

.980 

.975 

.968 

960 

.949 

.933 

.921 

.40 

998 

.996 

.994 

.990 

.987 

982 

.976 

968 

.958 

.940 

926 

.45 

.999 

998 

.996 

.994 

992 

.988 

.983 

.976 

966 

.947 

.930 

.50 

1.000 

1.000 

1.000 

1.000 

.995 

.993 

.989 

.984 

.974 

.955 

.935 

.55 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.994 

990 

982 

.962 

939 

.60 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.994 

.989 

.970 

943 

.65 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1000 

994 

978 

.946 

.70 

1.000 

1.000 

1000 

1.000 

1.000 

1.000 

1.000 

1.000 

998 

.985 

947 

.75 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

999 

.991 

947 

80 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

996 

947 

.85 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

999 

'■47 

.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

747 

.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.947 

1.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1  000 

1000 

947 

Proportion  of  employees  considered  satisfactory  -  .95 


m 

.950 

.950 

.950 

.950 

.950 

.950 

.950 

.950 

.950 

.950 

■gf 

.969 

.964 

.962 

.960 

.959 

.957 

956 

.954 

.952 

951 

mm 

.982 

.973 

.972 

.970 

.967 

.964 

.962 

.959 

.955 

.953 

.15 

.990 

.984 

.981 

.978 

.975 

.972 

.968 

.964 

.959 

.956 

.20 

.995 

MTfm 

.990 

.987 

.984 

.982 

.978 

974 

963 

959 

.25 

.998 

.996 

.994 

.992 

.990 

.987 

.984 

.975 

.968 

962 

.30 

.999 

.998 

.997 

.996 

.994 

.992 

.989 

.986 

.981 

.973 

.066 

.35 

1.000 

1.000 

.998 

.996 

.995 

.993 

.986 

.978 

970 

.40 

1.000 

1.000 

1.000 

1.000 

1.000 

.997 

.996 

.994 

.991 

.983 

475 

.45 

1.000 

1.000 

1.000 

1.000 

1.000 

.997 

.994 

988 

.479 

SO 

1.000 

1.000 

1.000 

1.000 

1.000 

1.0UU 

I.UUO 

.997 

992 

.984 

.55 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

t.OOU 

1.000 

.996 

.988 

.60 

uno 

1.000 

1.000 

1.000 

1.000 

■  K'X'M 

1.000 

1.000 

1.000 

.998 

.992 

.65 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.995 

.70 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.998 

.75 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.80 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000‘ 

1.000 

1.000 

1.000 

1.000 

.85 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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